Understanding approximate Fisher information for fast convergence of natural gradient descent in wide neural networks*
From MaRDI portal
Publication:5020052
Recommendations
- Efficient approximations of the fisher matrix in neural networks using kronecker product singular value decomposition
- A convergence analysis of Nesterov's accelerated gradient method in training deep linear neural networks
- Wide neural networks of any depth evolve as linear models under gradient descent *
- QNG: a quasi-natural gradient method for large-scale statistical learning
- Riemannian Natural Gradient Methods
Cites work
- Explicit inverses of some tridiagonal matrices
- scientific article; zbMATH DE number 7306852 (Why is no real title available?)
- On the discrete analogues of some generalizations of Gronwall's inequality
- Riemannian metrics for neural networks. I: Feedforward networks
- Tridiagonal Toeplitz matrices: properties and novel applications
- Wide neural networks of any depth evolve as linear models under gradient descent *
Cited in
(6)- Sketch-based empirical natural gradient methods for deep learning
- Pathological Spectra of the Fisher Information Metric and Its Variants in Deep Neural Networks
- SCORE: approximating curvature information under self-concordant regularization
- Eigenvalue-Corrected Natural Gradient Based on a New Approximation
- A convergence analysis of Nesterov's accelerated gradient method in training deep linear neural networks
- Efficient approximations of the fisher matrix in neural networks using kronecker product singular value decomposition
This page was built for publication: Understanding approximate Fisher information for fast convergence of natural gradient descent in wide neural networks*
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5020052)
