TAP free energy, spin glasses and variational inference

DOI10.1214/20-AOP1443zbMATH Open1467.60025arXiv1808.07890OpenAlexW3125529942MaRDI QIDQ2227708FDOQ2227708

Authors: Yanyan Li 

Publication date: 15 February 2021

Published in: The Annals of Probability (Search for Journal in Brave)

Abstract: We consider the Sherrington-Kirkpatrick model of spin glasses with ferromagnetically biased couplings. For a specific choice of the couplings mean, the resulting Gibbs measure is equivalent to the Bayesian posterior for a high-dimensional estimation problem known as `$Z_2$ synchronization'. Statistical physics suggests to compute the expectation with respect to this Gibbs measure (the posterior mean in the synchronization problem), by minimizing the so-called Thouless-Anderson-Palmer (TAP) free energy, instead of the mean field (MF) free energy. We prove that this identification is correct, provided the ferromagnetic bias is larger than a constant (i.e. the noise level is small enough in synchronization). Namely, we prove that the scaled $el{l}_2$ distance between any low energy local minimizers of the TAP free energy and the mean of the Gibbs measure vanishes in the large size limit. Our proof technique is based on upper bounding the expected number of critical points of the TAP free energy using the Kac-Rice formula.

Full work available at URL: https://arxiv.org/abs/1808.07890

Recommendations

• The TAP-Plefka variational principle for the spherical SK model
• On the TAP free energy in the mixed \(p\)-spin models
• The generalized TAP free energy. II
• Dynamical approach to the TAP equations for the Sherrington-Kirkpatrick model
• Mean-field theory for a spin-glass model of neural networks: TAP free energy and the paramagnetic to spin-glass transition

zbMATH Keywords

Bayesian inferenceKac-Rice formulafree probabilitySherrington-Kirkpatrick modelTAP complexity

Mathematics Subject Classification ID

Bayesian inference (62F15) Large deviations (60F10)

Cites Work

• Graphical models, exponential families, and variational inference
• Concentration inequalities. A nonasymptotic theory of independence
• Probabilistic graphical models.
• Local operator theory, random matrices and Banach spaces.
• The eigenvalues of random symmetric matrices
• The Markov chain Monte Carlo revolution
• An introduction to random matrices
• Title not available (Why is that?)
• Random Fields and Geometry
• Correlation clustering
• Large deviations techniques and applications.
• Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices
• Title not available (Why is that?)
• Measurable selections of extrema
• Complexity of random energy landscapes, glass transition, and absolute value of the spectral determinant of random matrices
• Spin glasses and Stein's method
• Broken replica symmetry bounds in the mean field spin glass model
• Orientability and diffusion maps
• Mean field models for spin glasses. Volume I: Basic examples.
• Limit laws for random matrices and free products
• Free convolution with a semicircular distribution and eigenvalues of spiked deformations of Wigner matrices
• On the free convolution with a semi-circular distribution
• An introduction to MCMC for machine learning
• Random matrices and complexity of spin glasses
• Information, Physics, and Computation
• The Dynamics of Message Passing on Dense Graphs, with Applications to Compressed Sensing
• An iterative construction of solutions of the TAP equations for the Sherrington-Kirkpatrick model
• Statistical Physics of Spin Glasses and Information Processing
• Angular synchronization by eigenvectors and semidefinite programming
• A Simple Approach to the Global Regime of Gaussian Ensembles of Random Matrices
• Complexity of random smooth functions on the high-dimensional sphere
• The complexity of spherical \(p\)-spin models: a second moment approach
• Fundamental limits of symmetric low-rank matrix estimation
• Markov chains and mixing times. With a chapter on ``Coupling from the past by James G. Propp and David B. Wilson.
• Phase transitions in semidefinite relaxations
• Semidefinite programs on sparse random graphs and their application to community detection
• Optimality and sub-optimality of PCA. I: Spiked random matrix models
• The landscape of the spiked tensor model
• Mean-field equations for spin models with orthogonal interaction matrices
• The zero set of a real analytic function
• Asymptotic mutual information for the balanced binary stochastic block model
• On the TAP free energy in the mixed \(p\)-spin models
• On the formal equivalence of the TAP and thermodynamic methods in the SK model
• Thouless-Anderson-Palmer equations for generic \(p\)-spin glasses
• The adaptive interpolation method: a simple scheme to prove replica formulas in Bayesian inference

Cited In (10)

• Landscape complexity beyond invariance and the elastic manifold
• On the GREM approximation of TAP free energies
• Sudakov-Fernique post-AMP, and a new proof of the local convexity of the TAP free energy
• On the concavity of the TAP free energy in the SK model
• Exponential growth of random determinants beyond invariance
• Local convexity of the TAP free energy and AMP convergence for \(\mathbb{Z}_2\)-synchronization
• The TAP-Plefka variational principle for the spherical SK model
• Triviality of the geometry of mixed \(p\)-spin spherical Hamiltonians with external field
• On the REM approximation of TAP free energies
• The TAP free energy for high-dimensional linear regression

This page was built for publication: TAP free energy, spin glasses and variational inference

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2227708)