Score-Based Generative Models Detect Manifolds

MaRDI QIDQ6400912arXivFDO

Authors Jakiw Pidstrigach

Publication date 2 June 2022

Published in arXiv

Mathematics Subject Classification ID

Artificial intelligence (68T99)

arXiv classification

cs.LGcs.NAmath.NAmath.PRstat.ML

Abstract: Score-based generative models (SGMs) need to approximate the scores $ablalog{p}_t$ of the intermediate distributions as well as the final distribution $p_T$ of the forward process. The theoretical underpinnings of the effects of these approximations are still lacking. We find precise conditions under which SGMs are able to produce samples from an underlying (low-dimensional) data manifold $mathcalM$. This assures us that SGMs are able to generate the "right kind of samples". For example, taking $mathcalM$ to be the subset of images of faces, we find conditions under which the SGM robustly produces an image of a face, even though the relative frequencies of these images might not accurately represent the true data generating distribution. Moreover, this analysis is a first step towards understanding the generalization properties of SGMs: Taking $mathcalM$ to be the set of all training samples, our results provide a precise description of when the SGM memorizes its training data.

Cited in

(1)

• Optimal Transport Bounds for Latent Density Smoothing on MTW($K>0$) Manifolds

This page was built for publication: Score-Based Generative Models Detect Manifolds

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6400912)