Wide neural networks of any depth evolve as linear models under gradient descent *
From MaRDI portal
Publication:5857449
Recommendations
- Universality of gradient descent neural network training
- Wide neural networks with bottlenecks are deep Gaussian processes
- Multilayer feedforward networks are universal approximators
- Linearized two-layers neural networks in high dimension
- Effects of depth, width, and initialization: a convergence analysis of layer-wise training for deep linear neural networks
- Gradient descent optimizes over-parameterized deep ReLU networks
- ReLU networks are universal approximators via piecewise linear or constant functions
- Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizers
- On the benefit of width for neural networks: disappearance of basins
- scientific article; zbMATH DE number 759415
Cites work
Cited in
(65)- Linearized two-layers neural networks in high dimension
- Machine unlearning: linear filtration for logit-based classifiers
- The interpolation phase transition in neural networks: memorization and generalization under lazy training
- Provably training overparameterized neural network classifiers with non-convex constraints
- Discriminative clustering with representation learning with any ratio of labeled to unlabeled data
- Surprises in high-dimensional ridgeless least squares interpolation
- Loss landscapes and optimization in over-parameterized non-linear systems and neural networks
- When and why PINNs fail to train: a neural tangent kernel perspective
- Improved architectures and training algorithms for deep operator networks
- Data science applications to string theory
- Landscape and training regimes in deep learning
- On the eigenvector bias of Fourier feature networks: from regression to solving multi-scale PDEs with physics-informed neural networks
- A statistician teaches deep learning
- Drop-activation: implicit parameter reduction and harmonious regularization
- Free dynamics of feature learning processes
- Affine-invariant ensemble transform methods for logistic regression
- Phase diagram for two-layer ReLU neural networks at infinite-width limit
- An analytic theory of shallow networks dynamics for hinge loss classification*
- Understanding approximate Fisher information for fast convergence of natural gradient descent in wide neural networks*
- On the Exact Computation of Linear Frequency Principle Dynamics and Its Generalization
- scientific article; zbMATH DE number 7626727 (Why is no real title available?)
- Align, then memorise: the dynamics of learning with feedback alignment*
- Asymptotics of representation learning in finite Bayesian neural networks*
- Locality defeats the curse of dimensionality in convolutional teacher–student scenarios*
- Adaptive and Implicit Regularization for Matrix Completion
- Wide neural networks with bottlenecks are deep Gaussian processes
- Plateau phenomenon in gradient descent training of RELU networks: explanation, quantification, and avoidance
- Generalization properties of hyper-RKHS and its applications
- The ridgelet prior: a covariance function approach to prior specification for Bayesian neural networks
- On the effect of the activation function on the distribution of hidden nodes in a deep network
- Multilevel Fine-Tuning: Closing Generalization Gaps in Approximation of Solution Maps under a Limited Budget for Training Data
- Training a Neural-Network-Based Surrogate Model for Aerodynamic Optimisation Using a Gaussian Process
- Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation
- Adversarial examples in random neural networks with general activations
- A rigorous framework for the mean field limit of multilayer neural networks
- Priors in Bayesian Deep Learning: A Review
- Universality of gradient descent neural network training
- Deep stable neural networks: large-width asymptotics and convergence rates
- Graph-based sparse Bayesian broad learning system for semi-supervised learning
- Weighted neural tangent kernel: a generalized and improved network-induced kernel
- Deep Q‐learning: A robust control approach
- An asynchronous parallel high-throughput model calibration framework for crystal plasticity finite element constitutive models
- On the spectral bias of coupled frequency predictor-corrector triangular DNN: the convergence analysis
- Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy
- Gradient descent on infinitely wide neural networks: global convergence and generalization
- Abstracting instance information and inter-label relations for sparse multi-label classification
- Learning effective stochastic differential equations from microscopic simulations: linking stochastic numerics to deep learning
- Infinite-width limit of deep linear neural networks
- Exact learning dynamics of deep linear networks with prior knowledge
- Precise learning curves and higher-order scaling limits for dot-product kernel regression
- Self-consistent dynamical field theory of kernel evolution in wide neural networks
- Redundant representations help generalization in wide neural networks
- MHDnet: physics-preserving learning for solving magnetohydrodynamics problems
- Autoregressive typical thermal states
- Optimizing the optimizer for physics-informed neural networks and Kolmogorov-Arnold networks
- Wide stable neural networks: sample regularity, functional convergence and Bayesian inverse problems
- Approximation results for gradient flow trained neural networks
- On the robustness of the minimim _2 interpolator
- Trained quantum neural networks are Gaussian processes
- Unraveling the Hessian: a key to smooth convergence in loss function landscapes
- On spectral bias reduction of multi-scale neural networks for regression problems
- Unveiling the optimization process of physics informed neural networks: how accurate and competitive can PINNs be?
- A universal law of robustness via isoperimetry
- Bayes-optimal learning of deep random networks of extensive-width
- Fundamental limits of overparametrized shallow neural networks for supervised learning
Describes a project that uses
Uses Software
This page was built for publication: Wide neural networks of any depth evolve as linear models under gradient descent *
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5857449)
