Statistical mechanics of learning
intelligent behaviorlearning rulesmultilayer networksneural networkson-line learningstatistical mechanics
Research exposition (monographs, survey articles) pertaining to computer science (68-02) Learning and adaptive systems in artificial intelligence (68T05) Research exposition (monographs, survey articles) pertaining to statistical mechanics (82-02) Neural nets applied to problems in time-dependent statistical mechanics (82C32) Neural networks for/in biological studies, artificial life and related topics (92B20)
In the framework of artificial neural networks the process of learning from examples can be specified. The authors provide various important concepts over the last decade by applying the techniques of statistical mechanics to the subject of learning processes. The book provides an introduction both to basic notions necessary to study learning processes, and techniques to obtain quantitative results. A major part deals with the perceptron as a fundamental instrument for constructing neural networks. The concepts are supplemented with background material in mathematics and physics, and contain numerous examples and exercises. While the basic computational techniques are presented in great detail, the more technical aspects have been collected in the appendices. The discussion section at the end of every chapter includes a brief review of the literature. Contents: 1. Getting Started (artificial neural networks, general setup); 2. Perceptron Learning -- Basics (Gibbs learning, the simulated annealed approximation, Gardner analysis); 3. A Choice of Learning Rules (Hebb rule, adaline rule, Bayes rule); 4. Augmented Statistical Mechanics Formulation (maximal stabilities, general statistical mechanics formulation, optimal potential); 5. Noisy Teachers (perfect learning, learning with errors); 6. The Storage Problem (Cover analysis, Galilean pastiche: Ising perceptron); 7. Discontinuous Learning (smooth networks, dynamics of discontinuous learning); 8. Unsupervised Learning (deceptions of randomness, clustering through competitive learning); 9. On-line Learning (perception with a smooth transfer function, unsupervised on-line learning); 10. Making Contact with Statistics (Sauer's lemma, Vapik-Chernonenkis theorem, comparison with statistical mechanics); 11. Multifractals (the multifractal organization of internal representations); 12. Multilayer Networks (parity tree, committee tree, committee machine); 13. On-line Learning in Multilayer Networks (Bayesian on-line learning); What Else? (support vector machines, complex optimization, error-correcting codes, game theory); Appendices. The book can be recommended both to students of the subjects artificial intelligence, statistics, of interdisciplinary subjects in psychology and philosophy, and to scientists and applied researchers interested in concepts of intelligent learning processes.
- Phase transitions and learning in neural networks
- Complexity of learning in artificial neural networks
- Neural networks: A review from a statistical perspective. With comments and a rejoinder by the authors
- Statistical mechanics of training in neural networks
- scientific article; zbMATH DE number 2211481
- Generalization learning in a perceptron with binary synapses
- Innovation rather than improvement: a solvable high-dimensional model highlights the limitations of scalar fitness
- Moral foundations in an interacting neural networks society: a statistical mechanics analysis
- Pursuit of food \textit{versus} pursuit of information in a Markovian perception-action loop model of foraging
- Optimal node perturbation in linear perceptrons with uncertain eligibility trace
- Generalized Guerra's interpolation schemes for dense associative neural networks
- Strategies for an efficient official publicity campaign
- High-dimensional dynamics of generalization error in neural networks
- Hidden unit specialization in layered neural networks: ReLU vs. sigmoidal activation
- A spin glass model for the loss surfaces of generative adversarial networks
- Why are macroscopic experiments reproducible? Imitating the behavior of an ensemble by single pure states
- On non-negative solutions to large systems of random linear equations
- An agent-based lattice model for the emergence of anti-microbial resistance
- Eigenspace stability conditions in mean-field replica theories
- Performance analysis of LVQ algorithms: a statistical physics approach
- \(p\)-adic statistical field theory and deep belief networks
- Study of phase transition of Potts model with domain adversarial neural network
- Phase transitions and learning in neural networks
- A numerical investigation of the jamming transition in traffic flow on diluted planar networks
- Expectation propagation with factorizing distributions: a Gaussian approximation and performance results for simple models
- Window-Based Example Selection in Learning Vector Quantization
- Neural cryptography with queries
- Stability of the replica symmetric solution in diluted perceptron learning
- Statistical mechanics of complex neural systems and high dimensional data
- Inferring hidden states in a random kinetic Ising model: replica analysis
- A Max-Sum algorithm for training discrete neural networks
- Learning of couplings for random asymmetric kinetic Ising models revisited: random correlation matrices and learning curves
- A statistical physics approach to learning curves for the inverse Ising problem
- Statistical mechanics of the inverse Ising problem and the optimal objective function
- The loss surfaces of neural networks with general activation functions
- Active online learning in the binary perceptron problem
- About the ergodic regime in the analogical Hopfield neural networks: Moments of the partition function
- Learning in the multilayer perceptron
- Dynamics of learning
- Learning dynamics: a replica approach
- Multistate perceptrons: learning rule and perceptron of maximal stability
- Learning from stochastic rules under finite temperature - optimal temperature and asymptotic learning curve
- Perfect learning and power law in learning from stochastic examples by Ising perceptrons: analysis under one-step replica symmetry breaking ansatz
- scientific article; zbMATH DE number 1172290 (Why is no real title available?)
- scientific article; zbMATH DE number 940827 (Why is no real title available?)
- Statistical mechanics of training in neural networks
- Triple descent and the two kinds of overfitting: where and why do they appear?*
- A random matrix analysis of random Fourier features: beyond the Gaussian kernel, a precise phase transition, and the corresponding double descent*
- Generalisation error in learning with random features and the hidden manifold model*
- Align, then memorise: the dynamics of learning with feedback alignment
- Learning curves of generic features maps for realistic datasets with a teacher-student model*
- Align, then memorise: the dynamics of learning with feedback alignment*
- Ising model selection using ℓ 1-regularized linear regression: a statistical mechanics analysis*
- An analytical theory of curriculum learning in teacher–student networks*
- Storage capacity in symmetric binary perceptrons
- Mean-field inference methods for neural networks
- Information theoretic limits of learning a sparse rule
- Dreaming neural networks: rigorous results
- Optimization of the Sherrington--Kirkpatrick Hamiltonian
- Forgetting Memories and Their Attractiveness
- Blessing of dimensionality: mathematical foundations of the statistical physics of data
- A new spin on optimal portfolios and ecological equilibria
- scientific article; zbMATH DE number 7415108 (Why is no real title available?)
- Node perturbation learning without noiseless baseline
- Anti-pairing in learning of a neural network with redundant hidden units
- scientific article; zbMATH DE number 2211481 (Why is no real title available?)
- Fluctuation-Dissipation Theorem and Models of Learning
- Replica theory for Levy spin glasses
- Self-planting: digging holes in rough landscapes
- Entropy and mutual information in models of deep neural networks*
- The committee machine: computational to statistical gaps in learning a two-layers neural network
- Clustering of solutions in the symmetric binary perceptron
- Scaling description of generalization with number of parameters in deep learning
- Asymptotic learning curves of kernel methods: empirical data versus teacher–student paradigm
- Dynamics of stochastic gradient descent for two-layer neural networks in the teacher–student setup*
- Wide flat minima and optimal generalization in classifying high-dimensional Gaussian mixtures
- Critical properties of the SAT/UNSAT transitions in the classification problem of structured data
- Replica symmetry breaking in neural networks: a few steps toward rigorous results
- Replica analysis of the lattice-gas restricted Boltzmann machine partition function
- A jamming transition from under- to over-parametrization affects generalization in deep learning
- Disordered high-dimensional optimal control
- Statistical physics of interacting neural networks
- Complexity of learning in artificial neural networks
- The emergence of a concept in shallow neural networks
- Perturbed anisotropic opinion dynamics with delayed information
- Dense Hebbian neural networks: a replica symmetric picture of supervised learning
- On the capacity of a quantum perceptron for storing biased patterns
- Introduction to the dynamics of disordered systems: equilibrium and gradient descent
- An introduction to machine learning: a perspective from statistical physics
- Hebbian dreaming for small datasets
- Replica symmetry breaking in supervised and unsupervised Hebbian networks
- Max-affine regression via first-order methods
- Dynamical mean field theory for models of confluent tissues and beyond
- Parallel learning by multitasking neural networks
- Hebbian learning from first principles
- A note on the dynamics of extended-context disordered kinetic spin models
- Neural networks trained with SGD learn distributions of increasing complexity^*
- High-dimensional learning of narrow neural networks
- Optimization of the Sherrington-Kirkpatrick Hamiltonian
- Exploring research trends in modern machine learning through a statistical mechanics lens: an Ising-inspired modeling perspective
- Feature learning in finite-width Bayesian deep linear networks with multiple outputs and convolutional layers
- Overview frequency principle/spectral bias in deep learning
- The generalization error of max-margin linear classifiers: benign overfitting and high dimensional asymptotics in the overparametrized regime
- Statistical inference in classification of high-dimensional Gaussian mixture
- Injectivity of ReLU networks: perspectives from statistical physics
This page was built for publication: Statistical mechanics of learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2723579)
