Introduction

In the ever-evolving landscape of Generative AI, we witness exponential growth, notably in topics like large-language models, text-to-image, video generation, multimodal reasoning, drug discovery, robotics, and beyond. Applied research thrives, yet it is imperative to spotlight foundational works.

Introducing the "Frontier Topics in Generative AI" Seminar Series. This series focuses on an in-depth exploration of grounded research in GenAI, including vision, language, and other pivotal areas. Join us in unraveling the core principles propelling innovation, shaping the future of this dynamic field.

Registration

To attend the seminar please register at: https://asu.zoom.us/meeting/register/tZ0scu2gqzkqH9AlBRhw4TBQBlU2a4ZsC_ZV

Schedule

Yilun Du
PhD Candidate, MIT

Title: Generalizing Outside the Training Distribution through Compositional Generation

Abstract: Generative AI has led to stunning successes in recent years but is fundamentally limited by the amount of data available. This is especially limiting in the embodied setting – where an agent must make decisions in completely new environments. In this talk, I’ll introduce the idea of compositional generative modeling, which can significantly reduce needed data requirements by building complex generative models from smaller constituents. I’ll first introduce the idea of energy-based models and illustrate how they enable compositional generative modeling. I’ll then illustrate how such compositional models enable the synthesis of complex plans in novel environments as well as complex visual scenes in unseen environments. Finally, I'll show how such compositionality can be applied to multimodal models to construct decision making systems that can hierarchically plan to solve long-horizon problems.

Huan Wang
PhD Candidate, Northeastern University

Title: Efficient Mobile Text-to-Image Diffusion Models

Abstract: Diffusion models (DMs) trained on enormous text-image pairs, such as Stable Diffusion, DALL-E, have revolutionized the field visual information generation with their exceptional quality. However, the superior quality of these models is offset by their substantial size and the consequent slow inference speed, a challenge that becomes even more pronounced on mobile devices. In this talk, I will first discuss the challenges of running text-to-image DMs on mobile devices. Then, I shall introduce our NeurIPS’23 work, “SnapFusion -- Text-to-Image Diffusion Model on Mobile Devices within Two Seconds”, which is known as the first approach that can achieve text-to-image generation in less than 2 seconds on a mobile device. Particularly, I shall explain how we significantly improve the inference efficiency through a joint optimization of the network architecture and training strategy. Other relevant works (e.g., Google's recent work, MobileDiffusion) will also be discussed. Finally, a summary and outlook of the mobile DMs in the future will conclude the talk.

Deqian Kong
PhD Candidate, University of California, Los Angeles

Title: Top-down Latent Space Generative Models for Planning and Optimization

Abstract: In this talk, we explore recent developments in latent space generative models, with a focus on our contributions to latent space energy-based models and latent plan transformers. We compare these models to current prevalent methods such as diffusion models and causal transformers, highlighting how our approach aims to offer explicit abstractions for improved generalization, planning, and online optimization. We will discuss two main applications of our work, the adaptation of offline reinforcement learning for planning purposes, and the use of our models in online optimization, particularly with a focus on molecule design and drug discovery. These applications are representative examples of how latent space models can be applied to complex problems, offering potential pathways for further research and exploration in these areas.

Introduction

Registration

Schedule

Organizers