Gradient descent optimizes over-parameterized deep ReLU networks
From MaRDI portal
Recommendations
- Non-convergence of stochastic gradient descent in the training of deep neural networks
- Complexity of training ReLU neural network
- A comparative analysis of optimization and generalization properties of two-layer neural network and random feature models under gradient descent dynamics
- The global optimization geometry of shallow linear neural networks
- Theoretical issues in deep networks
Cites work
Cited in
(83)- The global optimization geometry of shallow linear neural networks
- Global convergence and geometric characterization of slow to fast weight evolution in neural network training for classifying linearly non-separable data
- Non-convergence of stochastic gradient descent in the training of deep neural networks
- Linearized two-layers neural networks in high dimension
- Gradient convergence of deep learning-based numerical methods for BSDEs
- Normalization effects on shallow neural networks and related asymptotic expansions
- Stabilize deep ResNet with a sharp scaling factor \(\tau\)
- The interpolation phase transition in neural networks: memorization and generalization under lazy training
- Provably training overparameterized neural network classifiers with non-convex constraints
- Surprises in high-dimensional ridgeless least squares interpolation
- Generalization error of random feature and kernel methods: hypercontractivity and kernel matrix concentration
- Loss landscapes and optimization in over-parameterized non-linear systems and neural networks
- A proof of convergence for gradient descent in the training of artificial neural networks for constant target functions
- A proof of convergence for stochastic gradient descent in the training of artificial neural networks with ReLU activation for constant target functions
- Gradient descent optimizes over-parameterized deep ReLU networks
- A comparative analysis of optimization and generalization properties of two-layer neural network and random feature models under gradient descent dynamics
- Optimization for deep learning: an overview
- Piecewise convexity of artificial neural networks
- Towards interpreting deep neural networks via layer behavior understanding
- Growing axons: greedy learning of neural networks with application to function approximation
- Greedy training algorithms for neural networks and applications to PDEs
- Optimization in gradient networks
- The implicit bias of gradient descent on separable data
- Global minima of overparameterized neural networks
- Deep double descent: where bigger models and more data hurt*
- Memory Capacity of Neural Networks with Threshold and Rectified Linear Unit Activations
- Effects of depth, width, and initialization: a convergence analysis of layer-wise training for deep linear neural networks
- Revisiting Landscape Analysis in Deep Neural Networks: Eliminating Decreasing Paths to Infinity
- scientific article; zbMATH DE number 7625201 (Why is no real title available?)
- Particle dual averaging: optimization of mean field neural network with global convergence rate analysis*
- Benign overfitting in linear regression
- Full error analysis for the training of deep neural networks
- On the benefit of width for neural networks: disappearance of basins
- Gradient descent with identity initialization efficiently learns positive-definite linear transformations by deep residual networks
- Plateau phenomenon in gradient descent training of RELU networks: explanation, quantification, and avoidance
- When does gradient descent with logistic loss find interpolating two-layer networks?
- Dying ReLU and initialization: theory and numerical examples
- Effect of depth and width on local minima in deep learning
- Every Local Minimum Value Is the Global Minimum Value of Induced Model in Nonconvex Machine Learning
- On the effect of the activation function on the distribution of hidden nodes in a deep network
- Wide neural networks of any depth evolve as linear models under gradient descent *
- Dynamics of stochastic gradient descent for two-layer neural networks in the teacher–student setup*
- Suboptimal Local Minima Exist for Wide Neural Networks with Smooth Activations
- Deep learning: a statistical viewpoint
- Deep learning in random neural fields: numerical experiments via neural tangent kernel
- Non-differentiable saddle points and sub-optimal local minima exist for deep ReLU networks
- Black holes and the loss landscape in machine learning
- A rigorous framework for the mean field limit of multilayer neural networks
- A geometric approach of gradient descent algorithms in linear neural networks
- Overall error analysis for the training of deep neural networks via stochastic gradient descent with random initialisation
- Limitations of neural network training due to numerical instability of backpropagation
- Convergence rates for shallow neural networks learned by gradient descent
- On stochastic roundoff errors in gradient descent with low-precision computation
- FedHD: communication-efficient federated learning from hybrid data
- Normalization effects on deep neural networks
- Value iteration for streaming data on a continuous space with gradient method in an RKHS
- Global convergence in learning fully-connected ReLU networks via un-rectifying based on the augmented Lagrangian approach
- Gauss Newton method for solving variational problems of PDEs with neural network discretizaitons
- On the training and generalization of deep operator networks
- How many neurons do we need? A refined analysis for shallow networks trained with gradient descent
- Spatially regularized Leaky ReLU in dual space for CNN based image segmentation
- Strong overall error analysis for the training of artificial neural networks via random initializations
- Approximation results for gradient flow trained shallow neural networks in \(1d\)
- Gradient descent provably escapes saddle points in the training of shallow ReLU networks
- Assembly and iteration: transition to linearity of wide neural networks
- Approximation and gradient descent training with neural networks
- Training multi-layer over-parametrized neural network in subquadratic time
- On the rate of convergence of an over-parametrized deep neural network regression estimate with ReLU activation function learned by gradient descent
- Understanding the training of infinitely deep and wide ResNets with conditional optimal transport
- Towards an understanding of residual networks using neural tangent hierarchy (NTH)
- Random pruning over-parameterized neural networks can improve generalization: a training dynamics analysis
- Convergence of stochastic gradient descent under a local Łojasiewicz condition for deep neural networks
- Approximation results for gradient flow trained neural networks
- Beyond the quadratic approximation: the multiscale structure of neural network loss landscapes
- On the existence of global minima and convergence analyses for gradient descent methods in the training of deep neural networks
- Embedding principle: a hierarchical structure of loss landscape of deep neural networks
- On the convergence analysis of over-parameterized variational autoencoders: a neural tangent kernel perspective
- Efficient identification of wide shallow neural networks with biases
- Global convergence of SGD on two layer neural nets
- Six lectures on linearized neural networks
- A novel statistical approach to analyze image classification
- Curse of dimensionality in neural network optimization
- Almost sure convergence of dropout algorithms for neural networks
This page was built for publication: Gradient descent optimizes over-parameterized deep ReLU networks
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2183586)
