Statistical mechanics of complex neural systems and high dimensional data

DOI10.1088/1742-5468/2013/03/P03014zbMATH Open1456.82801arXiv1301.7115OpenAlexW3102632574MaRDI QIDQ3301560FDOQ3301560

Authors: Madhu S. Advani, Subhaneil Lahiri, Surya Ganguli 

Publication date: 11 August 2020

Published in: Journal of Statistical Mechanics: Theory and Experiment (Search for Journal in Brave)

Abstract: Recent experimental advances in neuroscience have opened new vistas into the immense complexity of neuronal networks. This proliferation of data challenges us on two parallel fronts. First, how can we form adequate theoretical frameworks for understanding how dynamical network processes cooperate across widely disparate spatiotemporal scales to solve important computational problems? And second, how can we extract meaningful models of neuronal systems from high dimensional datasets? To aid in these challenges, we give a pedagogical review of a collection of ideas and theoretical methods arising at the intersection of statistical physics, computer science and neurobiology. We introduce the interrelated replica and cavity methods, which originated in statistical physics as powerful ways to quantitatively analyze large highly heterogeneous systems of many interacting degrees of freedom. We also introduce the closely related notion of message passing in graphical models, which originated in computer science as a distributed algorithm capable of solving large inference and optimization problems involving many coupled variables. We then show how both the statistical physics and computer science perspectives can be applied in a wide diversity of contexts to problems arising in theoretical neuroscience and data analysis. Along the way we discuss spin glasses, learning theory, illusions of structure in noise, random matrices, dimensionality reduction, and compressed sensing, all within the unified formalism of the replica method. Moreover, we review recent conceptual connections between message passing in graphical models, and neural computation and learning. Overall, these ideas illustrate how statistical physics and computer science might provide a lens through which we can uncover emergent computational functions buried deep within the dynamical complexities of neuronal networks.

Full work available at URL: https://arxiv.org/abs/1301.7115

Recommendations

• Mean-field inference methods for neural networks
• Collective properties of neural networks: A statistical physics approach
• scientific article; zbMATH DE number 125396
• Mean field asymptotics in high-dimensional statistics: from exact results to efficient algorithms
• Statistical field theory for neural networks

Mathematics Subject Classification ID

Neural biology (92C20) Neural nets applied to problems in time-dependent statistical mechanics (82C32)

Cites Work

• An introduction to statistical modeling of extreme values
• Graphical models, exponential families, and variational inference
• Extensions of Lipschitz mappings into a Hilbert space
• An elementary proof of a theorem of Johnson and Lindenstrauss
• Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ ¹ minimization
• Probabilistic graphical models.
• Decoding by Linear Programming
• A simple proof of the restricted isometry property for random matrices
• Large deviations of the maximum eigenvalue in Wishart random matrices
• Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information
• From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images
• Title not available (Why is that?)
• Title not available (Why is that?)
• Level-spacing distributions and the Airy kernel
• Exact distribution of the maximal height of \(p\) vicious walkers
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Neural networks and physical systems with emergent collective computational abilities
• Sparse nonnegative solution of underdetermined linear equations by linear programming
• Neighborliness of randomly projected simplices in high dimensions
• Random projections of smooth manifolds
• Title not available (Why is that?)
• Constructing Free-Energy Approximations and Generalized Belief Propagation Algorithms
• A Brownian-Motion Model for the Eigenvalues of a Random Matrix
• Title not available (Why is that?)
• Nonintersecting Brownian excursions
• Gibbs states and the set of solutions of random constraint satisfaction problems
• On the distribution of the roots of certain symmetric matrices
• Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons.
• Information, Physics, and Computation
• Random matrix theory
• Statistical theory of superlattices
• Statistical Physics of Spin Glasses and Information Processing
• Compressed and Privacy-Sensitive Sparse Regression
• Survey propagation: An algorithm for satisfiability
• Statistical mechanics of learning
• The Perceptron: A Model for Brain Functioning. I
• Learning a rule in a multilayer neural network
• The space of interactions in neural network models
• Statistical mechanics of unsupervised structure recognition
• Observation of a hexatic vortex glass in flux lattices of the high-<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mrow><mml:msub><mml:mrow><mml:mi mathvariant="italic">T</mml:mi></mml:mrow><mml:mrow><mml:mi mathvariant="italic">c</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:math>superconductor<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mrow><mml:msub><mml:mrow><mml:mi mathvariant="normal">Bi</mml:mi></mml:mrow><mml:mrow><mml:…
• Role of homeostasis in learning sparse representations

Cited In (9)

• Statistical physics of community ecology: a cavity solution to MacArthur’s consumer resource model
• High-dimensional dynamics of generalization error in neural networks
• New statistical models of nonergodic cognitive systems and their pathologies
• Mean-field inference methods for neural networks
• Triple descent and the two kinds of overfitting: where and why do they appear?*
• Statistical mechanics of low-rank tensor decomposition
• Statistical physics and representations in real and artificial neural networks
• \(p\)-adic statistical field theory and deep belief networks
• Generalisation error in learning with random features and the hidden manifold model*

Uses Software

• ismev

This page was built for publication: Statistical mechanics of complex neural systems and high dimensional data

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3301560)