A spin glass model for the loss surfaces of generative adversarial networks

DOI10.1007/S10955-022-02875-WMaRDI QIDQ2076047zbMATH OpenOpenAlexFDO

Authors Nicholas P. Baskerville, J. P. Keating, F. Mezzadri, Joseph Najnudel

Publication date 18 February 2022

Published in Journal of Statistical Physics (Search for Journal in Brave)

Copyright license Creative Commons Attribution 4.0 International

Full work available at URL https://arxiv.org/abs/2101.02524

zbMATH Keywords

deep learninggenerative adversarial networksneural networksrandom matrix theoryspin glasses

Mathematics Subject Classification ID

Special matrices (15Bxx) Time-dependent statistical mechanics (dynamic and nonequilibrium) (82Cxx) Applications of statistical mechanics to specific types of physical systems (82Dxx)

Cites work

• Aging of spherical spin glasses
• Comparing dynamics: deep neural networks versus glassy systems
• Complexity of random energy landscapes, glass transition, and absolute value of the spectral determinant of random matrices
• Concentration of the spectral measure for large matrices
• Deep learning
• Dyson’s correlation functions and graded symmetry
• Entropy-SGD: biasing gradient descent into wide valleys
• Flat Minima
• Geometry and temperature chaos in mixed spherical spin glasses at low temperature: the perturbative regime
• Nonlinear random matrix theory for deep learning
• Random Fields and Geometry
• Random matrices and complexity of spin glasses
• Replica symmetry breaking condition exposed by random matrix calculation of landscape complexity
• Shaping the learning landscape in neural networks around wide flat minima
• Statistical mechanics of learning
• Statistical Physics of Spin Glasses and Information Processing
• The complexity of spherical p-spin models: a second moment approach
• The landscape of the spiked tensor model
• The loss surfaces of neural networks with general activation functions
• The space of interactions in neural network models

Cited in

(7)

• Generative adversarial networks are special cases of artificial curiosity (1990) and also closely related to predictability minimization (1991)
• Loss-sensitive generative adversarial networks on Lipschitz densities
• Exponential growth of random determinants beyond invariance
• Universal characteristics of deep neural network loss surfaces from random matrix theory
• Landscape complexity beyond invariance and the elastic manifold
• Local convexity of the TAP free energy and AMP convergence for \(\mathbb{Z}_2\)-synchronization
• A spin-glass model for the loss surfaces of generative adversarial networks

Describes a project that uses

Uses Software

• StyleGAN
• CIFAR
• Entropy-SGD
• Wasserstein GAN
• CycleGAN
• pix2pix
• BERT
• GitHub

This page was built for publication: A spin glass model for the loss surfaces of generative adversarial networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2076047)