Minimax formula for the replica symmetric free energy of deep restricted Boltzmann machines
From MaRDI portal
Publication:6104028
Abstract: We study the free energy of a most used deep architecture for restricted Boltzmann machines, where the layers are disposed in series. Assuming independent Gaussian distributed random weights, we show that the error term in the so-called replica symmetric sum rule can be optimised as a saddle point. This leads us to conjecture that in the replica symmetric approximation the free energy is given by a min max formula, which parallels the one achieved for two-layer case.
Recommendations
- A transport equation approach for deep neural networks with quenched random weights
- Annealing and replica-symmetry in deep Boltzmann machines
- Deep Boltzmann machines: rigorous results at arbitrary depth
- A dynamical mean-field theory for learning in restricted Boltzmann machines
- A Morita type proof of the replica-symmetric formula for SK
Cites work
- A remark on the spherical bipartite spin glass
- An efficient learning procedure for deep Boltzmann machines
- Annealing and replica-symmetry in deep Boltzmann machines
- Deep Boltzmann machines: rigorous results at arbitrary depth
- Equilibrium statistical mechanics of bipartite spin systems
- Free energy and complexity of spherical bipartite models
- Free energy of bipartite spherical Sherrington-Kirkpatrick model
- scientific article; zbMATH DE number 3560419 (Why is no real title available?)
- scientific article; zbMATH DE number 1751767 (Why is no real title available?)
- Legendre equivalences of spherical Boltzmann machines
- Mean field models for spin glasses. Volume I: Basic examples.
- Multi-species mean field spin glasses. Rigorous results
- The adaptive interpolation method: a simple scheme to prove replica formulas in Bayesian inference
- The free energy in a multi-species Sherrington-Kirkpatrick model
- The replica symmetric approximation of the analogical neural network
- The Sherrington-Kirkpatrick model: A challenge for mathematicians
- The solution of the deep Boltzmann machine on the Nishimori line
- Training Products of Experts by Minimizing Contrastive Divergence
- Universality in bipartite mean field spin glasses
Cited in
(10)- Annealing and replica-symmetry in deep Boltzmann machines
- Free energy computations by minimization of Kullback-Leibler divergence: An efficient adaptive biasing potential method for sparse representations
- Free energies of Boltzmann machines: self-averaging, annealed and replica symmetric approximations in the thermodynamic limit
- A transport equation approach for deep neural networks with quenched random weights
- Legendre equivalences of spherical Boltzmann machines
- Replica analysis of the lattice-gas restricted Boltzmann machine partition function
- Thouless-Anderson-Palmer equations for the multi-species Sherrington-Kirkpatrick model
- On the free energy of vector spin glasses with nonconvex interactions
- Free energy fluctuations of the bipartite spherical SK model at critical temperature
- On free energy of non-convex multi-species spin glasses
This page was built for publication: Minimax formula for the replica symmetric free energy of deep restricted Boltzmann machines
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6104028)
