A geometric modeling of Occam's razor in deep learning

DOI10.1007/S41884-025-00167-2MaRDI QIDQ7267409zbMATH OpenFDO

Authors Ke Sun, Frank Nielsen

Publication date 16 June 2026

Published in Information Geometry (Search for Journal in Brave)

Copyright license Creative Commons Attribution 4.0 International

Full work available at URL https://doi.org/10.1007/S41884-025-00167-2

zbMATH Keywords

deep learningFisher informationinformation geometryminimum description lengthstochastic complexity

Mathematics Subject Classification ID

Sufficiency and information (62Bxx) Artificial intelligence (68Txx) Communication, information (94Axx)

Cites work

• <tex>$alpha$</tex>-Parallel Prior and its Properties
• A new look at the statistical model identification
• A note on insufficiency and the preservation of Fisher information
• A review on unique existence theorems in lightlike geometry
• A tight excess risk bound via a unified PAC-Bayesian-Rademacher-Shtarkov-MDL complexity
• Algebraic Geometry and Statistical Learning Theory
• An Information Measure for Classification
• Counting probability distributions: Differential geometry and model selection
• Deep learning
• Deep learning architectures. A mathematical approach
• Dynamics of learning in MLP: natural gradient and singularity revisited
• Dynamics of Learning Near Singularities in Layered Networks
• Estimating the dimension of a model
• Fisher information and stochastic complexity
• Flat Minima
• Free probability and random matrices
• Gaussian Distributions on Riemannian Symmetric Spaces: Statistical Learning With Structured Covariance Matrices
• Geometric Modeling in Probability and Statistics
• scientific article; zbMATH DE number 4095371 (Why is no real title available?)
• scientific article; zbMATH DE number 44578 (Why is no real title available?)
• scientific article; zbMATH DE number 790016 (Why is no real title available?)
• scientific article; zbMATH DE number 868957 (Why is no real title available?)
• scientific article; zbMATH DE number 886054 (Why is no real title available?)
• scientific article; zbMATH DE number 7306852 (Why is no real title available?)
• scientific article; zbMATH DE number 3106666 (Why is no real title available?)
• Information geometry and its applications
• Information-theoretic upper and lower bounds for statistical estimation
• Large deviation theory for non-regular location shift family
• Minimum complexity density estimation
• Minimum description length revisited
• Modeling by shortest data description
• On the category of stratifolds
• PAC-MDL bounds.
• Pathological Spectra of the Fisher Information Metric and Its Variants in Deep Neural Networks
• Singularities in mixture models and upper bounds of stochastic complexity.
• Spaces of statistical parameters.
• Statistical mechanical analysis of learning dynamics of two-layer perceptron with multiple output units
• Strong optimality of the normalized ML models as universal codes and information in data
• Sur la notion de la moyenne.
• The dually flat structure for singular models
• The minimum description length principle in coding and modeling
• The rank of a random matrix
• Traces of class/cross-class structure pervade deep learning spectra
• Über eine Klasse der Mittelwerte.

This page was built for publication: A geometric modeling of Occam's razor in deep learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q7267409)