A tale of three probabilistic families: discriminative, descriptive, and generative models
From MaRDI portal
Probability distributions: general theory (60E05) Image analysis in multivariate analysis (62H35) Random fields; image analysis (62M40) Applications of statistics to biology and medical sciences; meta analysis (62P10) Neural networks for/in biological studies, artificial life and related topics (92B20)
Abstract: The pattern theory of Grenander is a mathematical framework where patterns are represented by probability models on random variables of algebraic structures. In this paper, we review three families of probability models, namely, the discriminative models, the descriptive models, and the generative models. A discriminative model is in the form of a classifier. It specifies the conditional probability of the class label given the input signal. A descriptive model specifies the probability distribution of the signal, based on an energy function defined on the signal. A generative model assumes that the signal is generated by some latent variables via a transformation. We shall review these models within a common framework and explore their connections. We shall also review the recent developments that take advantage of the high approximation capacities of deep neural networks.
Recommendations
Cites work
- 10.1162/jmlr.2003.4.7-8.1235
- A Connection Between Score Matching and Denoising Autoencoders
- A decision-theoretic generalization of on-line learning and an application to boosting
- A Fast Learning Algorithm for Deep Belief Nets
- A Stochastic Approximation Method
- A stochastic grammar of images
- A theory of preattentive texture discrimination based on first-order statistics of textons
- An efficient learning procedure for deep Boltzmann machines
- An Introduction to Variational Autoencoders
- Atomic Decomposition by Basis Pursuit
- Composition systems
- Deep learning
- EM algorithms for ML factor analysis
- Equivalence of Julesz ensembles and FRAME models
- Estimation of non-normalized statistical models by score matching
- Generative modeling of convolutional neural networks
- scientific article; zbMATH DE number 5813526 (Why is no real title available?)
- scientific article; zbMATH DE number 3860199 (Why is no real title available?)
- scientific article; zbMATH DE number 4143202 (Why is no real title available?)
- scientific article; zbMATH DE number 3513115 (Why is no real title available?)
- scientific article; zbMATH DE number 3567782 (Why is no real title available?)
- scientific article; zbMATH DE number 1560711 (Why is no real title available?)
- scientific article; zbMATH DE number 2140075 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- Learning active basis models by EM-type algorithms
- MCMC using Hamiltonian dynamics
- Modeling visual patterns by integrating descriptive and generative methods
- Monte Carlo strategies in scientific computing.
- Multivariate adaptive regression splines
- Neural networks and physical systems with emergent collective computational abilities
- On the convergence of markovian stochastic algorithms with rapidly decreasing ergodicity rates
- Pattern theory. From representation to inference.
- Pattern theory. The stochastic analysis of real-world signals.
- Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
- Support-vector networks
- Training Products of Experts by Minimizing Contrastive Divergence
- Unsupervised learning of compositional sparse code for natural image representation
- What regularized auto-encoders learn from the data-generating distribution
Describes a project that uses
Uses Software
This page was built for publication: A tale of three probabilistic families: discriminative, descriptive, and generative models
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3121222)
