A dynamical mean-field theory for learning in restricted Boltzmann machines
From MaRDI portal
Publication:5857421
DOI10.1088/1742-5468/ABB8C9zbMATH Open1459.82167arXiv2005.01560OpenAlexW3095006753MaRDI QIDQ5857421FDOQ5857421
Authors: Burak Çakmak, Manfred Opper 
Publication date: 1 April 2021
Published in: Journal of Statistical Mechanics: Theory and Experiment (Search for Journal in Brave)
Abstract: We define a message-passing algorithm for computing magnetizations in Restricted Boltzmann machines, which are Ising models on bipartite graphs introduced as neural network models for probability distributions over spin configurations. To model nontrivial statistical dependencies between the spins' couplings, we assume that the rectangular coupling matrix is drawn from an arbitrary bi-rotation invariant random matrix ensemble. Using the dynamical functional method of statistical mechanics we exactly analyze the dynamics of the algorithm in the large system limit. We prove the global convergence of the algorithm under a stability criterion and compute asymptotic convergence rates showing excellent agreement with numerical simulations.
Full work available at URL: https://arxiv.org/abs/2005.01560
Recommendations
- Thermodynamics of restricted Boltzmann machines and related learning dynamics
- Restricted Boltzmann machines: introduction and review
- Training restricted Boltzmann machines: an introduction
- Statistical mechanics of unsupervised feature learning in a restricted Boltzmann machine with binary synapses
- Learning restricted Boltzmann machines via influence maximization
Cites Work
- Title not available (Why is that?)
- The planar approximation. II
- New scaling of Itzykson-Zuber integrals
- Random matrix methods for wireless communications.
- Training Products of Experts by Minimizing Contrastive Divergence
- Information, Physics, and Computation
- New method for studying the dynamics of disordered spin systems without finite-size effects
- The Dynamics of Message Passing on Dense Graphs, with Applications to Compressed Sensing
- An iterative construction of solutions of the TAP equations for the Sherrington-Kirkpatrick model
- Statistical Physics of Spin Glasses and Information Processing
- Title not available (Why is that?)
- Rectangular \(R\)-transform as the limit of rectangular spherical integrals
- The space of interactions in neural network models
- Integration of invariant matrices and moments of inverses of Ginibre and Wishart matrices
- Introduction to Random Matrices
- Thermodynamics of restricted Boltzmann machines and related learning dynamics
- A theory of solving TAP equations for Ising models with general invariant random matrices
- Mean-field inference methods for neural networks
- Vector Approximate Message Passing
- High-temperature expansions and message passing algorithms
- Rigorous Dynamics of Expectation-Propagation-Based Signal Recovery from Unitarily Invariant Measurements
- Analysis of Bayesian inference algorithms by the dynamical functional approach
Cited In (12)
- Boltzmann Machine and Mean-Field Approximation for Structured Sparse Decompositions
- Learning and Inference in Sparse Coding Models With Langevin Dynamics
- Approximate message passing algorithms for rotationally invariant matrices
- Analysis of Bayesian inference algorithms by the dynamical functional approach
- Minimax formula for the replica symmetric free energy of deep restricted Boltzmann machines
- Universality of approximate message passing algorithms and tensor networks
- Gaussian-spherical restricted Boltzmann machines
- Dynamical analysis of contrastive divergence learning: restricted Boltzmann machines with Gaussian visible units
- Stochastic complexity and generalization error of a restricted Boltzmann machine in Bayesian estimation
- Analysis of random sequential message passing algorithms for approximate inference
- ‘Place-cell’ emergence and learning of invariant data with restricted Boltzmann machines: breaking and dynamical restoration of continuous symmetries in the weight space
- Learning large \(Q\)-matrix by restricted Boltzmann machines
This page was built for publication: A dynamical mean-field theory for learning in restricted Boltzmann machines
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5857421)
