March 2024

Date: 4PM Friday 8th March 2024

Transporting measures for sampling: parametric and non-parametric approaches inspired by generative modelling

Marylou Gabrié CMAP, École Polytechnique

Generative models and statistical mechanics have a long history of cross-fertilization. Recently, it has been shown that generative models, such as normalizing flows, can assist sampling of metastable systems. This remarkable ability comes from the high-expressivity of generative models that can approach complex distributions while remaining tractable. However, the training accuracy of generative models deteriorates when the dimension and complexity of the target measure are pushed. Inspired by recent progress in generative modelling relying on stochastic processes, non-parametric sampling algorithms can also be derived to sample from metastable systems. Are non-parametric methods more scalable?

Implicit Diffusion: Efficient Optimization through Stochastic Sampling

Pierre Marion École Polytechnique Fédérale de Lausanne

We present a new algorithm to optimize distributions defined implicitly by parameterized stochastic diffusions. Doing so allows us to modify the outcome distribution of sampling processes by optimizing over their parameters. We introduce a general framework for first-order optimization of these processes, that performs jointly, in a single loop, optimization and sampling steps. This approach is inspired by recent advances in bilevel optimization and automatic implicit differentiation, leveraging the point of view of sampling as optimization over the space of probability distributions. We provide theoretical guarantees on the performance of our method, as well as experimental results demonstrating its effectiveness in real-world settings.