Statistical Mechanics of Learning

DOI10.1017/CBO9781139164542zbMath0984.82034MaRDI QIDQ2723579

Andreas Engel, Christian Van den Broeck

Publication date: 8 July 2001

Full work available at URL: https://doi.org/10.1017/cbo9781139164542

zbMATH Keywords

neural networks; statistical mechanics; on-line learning; multilayer networks; learning rules; intelligent behavior

Mathematics Subject Classification ID

68T05: Learning and adaptive systems in artificial intelligence

92B20: Neural networks for/in biological studies, artificial life and related topics

82-02: Research exposition (monographs, survey articles) pertaining to statistical mechanics

82C32: Neural nets applied to problems in time-dependent statistical mechanics

68-02: Research exposition (monographs, survey articles) pertaining to computer science

Related Items

Unnamed Item, Triple descent and the two kinds of overfitting: where and why do they appear?*, A random matrix analysis of random Fourier features: beyond the Gaussian kernel, a precise phase transition, and the corresponding double descent*, Generalisation error in learning with random features and the hidden manifold model*, Align, then memorise: the dynamics of learning with feedback alignment*, Align, then memorise: the dynamics of learning with feedback alignment*, Learning curves of generic features maps for realistic datasets with a teacher-student model*, Ising model selection using ℓ ₁-regularized linear regression: a statistical mechanics analysis*, An analytical theory of curriculum learning in teacher–student networks*, Storage capacity in symmetric binary perceptrons, Mean-field inference methods for neural networks, Information theoretic limits of learning a sparse rule, Dreaming neural networks: rigorous results, Optimization of the Sherrington--Kirkpatrick Hamiltonian, Forgetting Memories and Their Attractiveness, Blessing of dimensionality: mathematical foundations of the statistical physics of data, A new spin on optimal portfolios and ecological equilibria, Statistical physics of interacting neural networks, Complexity of learning in artificial neural networks, Replica theory for Levy spin glasses, Self-planting: digging holes in rough landscapes, Entropy and mutual information in models of deep neural networks*, The committee machine: computational to statistical gaps in learning a two-layers neural network, Clustering of solutions in the symmetric binary perceptron, Scaling description of generalization with number of parameters in deep learning, Asymptotic learning curves of kernel methods: empirical data versus teacher–student paradigm, Dynamics of stochastic gradient descent for two-layer neural networks in the teacher–student setup*, Wide flat minima and optimal generalization in classifying high-dimensional Gaussian mixtures, Critical properties of the SAT/UNSAT transitions in the classification problem of structured data, Replica symmetry breaking in neural networks: a few steps toward rigorous results, Replica analysis of the lattice-gas restricted Boltzmann machine partition function, A jamming transition from under- to over-parametrization affects generalization in deep learning, Disordered high-dimensional optimal control, The emergence of a concept in shallow neural networks, Perturbed anisotropic opinion dynamics with delayed information, Dense Hebbian neural networks: a replica symmetric picture of supervised learning, Node perturbation learning without noiseless baseline, When are networks truly modular?, Generalization learning in a perceptron with binary synapses, Innovation rather than improvement: a solvable high-dimensional model highlights the limitations of scalar fitness, Moral foundations in an interacting neural networks society: a statistical mechanics analysis, Pursuit of food \textit{versus} pursuit of information in a Markovian perception-action loop model of foraging, Optimal node perturbation in linear perceptrons with uncertain eligibility trace, Generalized Guerra's interpolation schemes for dense associative neural networks, Strategies for an efficient official publicity campaign, High-dimensional dynamics of generalization error in neural networks, Hidden unit specialization in layered neural networks: ReLU vs. sigmoidal activation, A spin glass model for the loss surfaces of generative adversarial networks, Why are macroscopic experiments reproducible? Imitating the behavior of an ensemble by single pure states, On non-negative solutions to large systems of random linear equations, An agent-based lattice model for the emergence of anti-microbial resistance, Eigenspace stability conditions in mean-field replica theories, Performance analysis of LVQ algorithms: a statistical physics approach, \(p\)-adic statistical field theory and deep belief networks, Study of phase transition of Potts model with domain adversarial neural network, A NUMERICAL INVESTIGATION OF THE JAMMING TRANSITION IN TRAFFIC FLOW ON DILUTED PLANAR NETWORKS, Expectation Propagation with Factorizing Distributions: A Gaussian Approximation and Performance Results for Simple Models, Window-Based Example Selection in Learning Vector Quantization, Neural cryptography with queries, Stability of the replica symmetric solution in diluted perceptron learning, Statistical mechanics of complex neural systems and high dimensional data, Inferring hidden states in a random kinetic Ising model: replica analysis, A Max-Sum algorithm for training discrete neural networks, Learning of couplings for random asymmetric kinetic Ising models revisited: random correlation matrices and learning curves, A statistical physics approach to learning curves for the inverse Ising problem, Statistical mechanics of the inverse Ising problem and the optimal objective function, The loss surfaces of neural networks with general activation functions, Active Online Learning in the Binary Perceptron Problem, About the ergodic regime in the analogical Hopfield neural networks: Moments of the partition function