Barcodes as summary of loss function topology

DOI10.1134/S1064562423701570MaRDI QIDQ6124396zbMATH OpenOpenAlexFDO

Authors Serguei Barannikov, A. A. Korotin, D. A. Oganesyan, D. I. Emtsev, E. V. Burnaev

Publication date 27 March 2024

Published in Doklady Mathematics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1912.00043

zbMATH Keywords

persistent homologyneural networksMorse theorypersistence barcodesloss surface

Mathematics Subject Classification ID

Persistent homology and applications, topological data analysis (55N31)

Abstract: We apply the canonical forms (barcodes) of gradient Morse complexes to explore topology of loss surfaces. We present a novel algorithm for calculations of the objective function's barcodes of local minima. We have conducted experiments for calculating barcodes of local minima for benchmark functions and for loss surfaces of neural networks. Our experiments confirm two principal observations for loss surfaces of neural networks. First, the barcodes of local minima are located in a small lower part of the range of values of loss function of neural networks. Second, increase of the neural network's depth brings down the barcodes of local minima. This has natural implications for the neural network learning and the generalization ability.

Recommendations

• Learning representations of persistence barcodes
• 1-dimensional topological invariants to estimate loss surface non-convexity
• A framework for differential calculus on persistence barcodes
• The loss surfaces of neural networks with general activation functions
• Archetypal landscapes for deep neural networks

Cites work

• scientific article; zbMATH DE number 732198 (Why is no real title available?)
• scientific article; zbMATH DE number 6276119 (Why is no real title available?)
• scientific article; zbMATH DE number 3052530 (Why is no real title available?)
• A literature survey of benchmark functions for global optimisation problems
• Barcodes and area-preserving homeomorphisms
• Differentiable dynamical systems
• Entropy-SGD: biasing gradient descent into wide valleys
• Geometry helps in bottleneck matching and related problems
• Lectures on Morse theory, old and new
• Precise Arrhenius law for \(p\)-forms: the Witten Laplacian and Morse-Barannikov complex
• Scalar field analysis over point cloud data
• Stability of persistence diagrams

Cited in

(2)

• Functional loops: monitoring functional organization of deep neural networks using algebraic topology
• 1-dimensional topological invariants to estimate loss surface non-convexity

This page was built for publication: Barcodes as summary of loss function topology

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6124396)