TAP free energy, spin glasses and variational inference

DOI10.1214/20-AOP1443MaRDI QIDQ2227708zbMATH OpenOpenAlexFDO

Authors Yanyan Li

Publication date 15 February 2021

Published in The Annals of Probability (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1808.07890, https://projecteuclid.org/euclid.aop/1611306374

zbMATH Keywords

Bayesian inferenceKac-Rice formulafree probabilitySherrington-Kirkpatrick modelTAP complexity

Mathematics Subject Classification ID

Bayesian inference (62F15) Large deviations (60F10)

Abstract: We consider the Sherrington-Kirkpatrick model of spin glasses with ferromagnetically biased couplings. For a specific choice of the couplings mean, the resulting Gibbs measure is equivalent to the Bayesian posterior for a high-dimensional estimation problem known as `$Z_2$ synchronization'. Statistical physics suggests to compute the expectation with respect to this Gibbs measure (the posterior mean in the synchronization problem), by minimizing the so-called Thouless-Anderson-Palmer (TAP) free energy, instead of the mean field (MF) free energy. We prove that this identification is correct, provided the ferromagnetic bias is larger than a constant (i.e. the noise level is small enough in synchronization). Namely, we prove that the scaled $el{l}_2$ distance between any low energy local minimizers of the TAP free energy and the mean of the Gibbs measure vanishes in the large size limit. Our proof technique is based on upper bounding the expected number of critical points of the TAP free energy using the Kac-Rice formula.

Recommendations

• The TAP-Plefka variational principle for the spherical SK model
• On the TAP free energy in the mixed p-spin models
• The generalized TAP free energy. II
• Dynamical approach to the TAP equations for the Sherrington-Kirkpatrick model
• Mean-field theory for a spin-glass model of neural networks: TAP free energy and the paramagnetic to spin-glass transition

Cites work

• scientific article; zbMATH DE number 420886 (Why is no real title available?)
• scientific article; zbMATH DE number 1273988 (Why is no real title available?)
• A Simple Approach to the Global Regime of Gaussian Ensembles of Random Matrices
• An introduction to MCMC for machine learning
• An introduction to random matrices
• An iterative construction of solutions of the TAP equations for the Sherrington-Kirkpatrick model
• Angular synchronization by eigenvectors and semidefinite programming
• Asymptotic mutual information for the balanced binary stochastic block model
• Broken replica symmetry bounds in the mean field spin glass model
• Complexity of random energy landscapes, glass transition, and absolute value of the spectral determinant of random matrices
• Complexity of random smooth functions on the high-dimensional sphere
• Concentration inequalities. A nonasymptotic theory of independence
• Correlation clustering
• Free convolution with a semicircular distribution and eigenvalues of spiked deformations of Wigner matrices
• Fundamental limits of symmetric low-rank matrix estimation
• Graphical models, exponential families, and variational inference
• Information, Physics, and Computation
• Large deviations techniques and applications.
• Limit laws for random matrices and free products
• Local operator theory, random matrices and Banach spaces.
• Markov chains and mixing times. With a chapter on ``Coupling from the past by James G. Propp and David B. Wilson.
• Mean field models for spin glasses. Volume I: Basic examples.
• Mean-field equations for spin models with orthogonal interaction matrices
• Measurable selections of extrema
• On the TAP free energy in the mixed p-spin models
• On the formal equivalence of the TAP and thermodynamic methods in the SK model
• On the free convolution with a semi-circular distribution
• Optimality and sub-optimality of PCA. I: Spiked random matrix models
• Orientability and diffusion maps
• Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices
• Phase transitions in semidefinite relaxations
• Probabilistic graphical models.
• Random Fields and Geometry
• Random matrices and complexity of spin glasses
• Semidefinite programs on sparse random graphs and their application to community detection
• Spin glasses and Stein's method
• Statistical Physics of Spin Glasses and Information Processing
• The Dynamics of Message Passing on Dense Graphs, with Applications to Compressed Sensing
• The Markov chain Monte Carlo revolution
• The adaptive interpolation method: a simple scheme to prove replica formulas in Bayesian inference
• The complexity of spherical p-spin models: a second moment approach
• The eigenvalues of random symmetric matrices
• The landscape of the spiked tensor model
• The zero set of a real analytic function
• Thouless-Anderson-Palmer equations for generic \(p\)-spin glasses

Cited in

(10)

• Landscape complexity beyond invariance and the elastic manifold
• On the GREM approximation of TAP free energies
• Sudakov-Fernique post-AMP, and a new proof of the local convexity of the TAP free energy
• Exponential growth of random determinants beyond invariance
• On the concavity of the TAP free energy in the SK model
• Local convexity of the TAP free energy and AMP convergence for \(\mathbb{Z}_2\)-synchronization
• The TAP-Plefka variational principle for the spherical SK model
• Triviality of the geometry of mixed \(p\)-spin spherical Hamiltonians with external field
• On the REM approximation of TAP free energies
• The TAP free energy for high-dimensional linear regression

This page was built for publication: TAP free energy, spin glasses and variational inference

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2227708)