Globally Convergent Multilevel Training of Deep Residual Networks

DOI10.1137/21m1434076zbMath1515.65166arXiv2107.07572OpenAlexW3186990848MaRDI QIDQ6108152

Rolf H. Krause, Alena Kopaničáková

Publication date: 29 June 2023

Published in: SIAM Journal on Scientific Computing (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2107.07572

zbMATH Keywords

training algorithmdeep residual networksmultilevel minimizationtrustregion methods

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Numerical optimization and variational techniques (65K10) Multigrid methods; domain decomposition for initial value and initial-boundary value problems involving PDEs (65M55)

Related Items (1)

Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Training

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• On solving L-SR1 trust-region subproblems
• On the power of small-depth threshold circuits
• A Levenberg-Marquardt method for large nonlinear least-squares problems with dynamic accuracy in functions and gradients
• Stochastic optimization using a trust-region method and random models
• A recursive multilevel trust region method with application to fully monolithic phase-field models of brittle fracture
• Newton-type methods for non-convex optimization under inexact Hessian information
• Adaptive multilevel trust-region methods for time-dependent PDE-constrained optimization
• A proposal on machine learning via dynamical systems
• Adaptive Multilevel Inexact SQP Methods for PDE-Constrained Optimization
• A recursive Formula-trust-region method for bound-constrained nonlinear optimization
• On the Convergence of Recursive Trust-Region Methods for Multiscale Nonlinear Optimization and Applications to Nonlinear Mechanics
• Recursive Trust-Region Methods for Multiscale Nonlinear Optimization
• Updating Quasi-Newton Matrices with Limited Storage
• Multi-Level Adaptive Solutions to Boundary-Value Problems
• Trust Region Methods
• A Multigrid Tutorial, Second Edition
• A multigrid approach to discretized optimization problems
• Complexity and global rates of trust-region methods based on probabilistic models
• Adaptive Sampling Strategies for Stochastic Optimization
• Optimization Methods for Large-Scale Machine Learning
• A robust multi-batch L-BFGS method for machine learning
• Layer-Parallel Training of Deep Residual Neural Networks
• On a multilevel Levenberg–Marquardt method for the training of artificial neural networks and its application to the solution of partial differential equations
• Quasi-Newton methods for machine learning: forget the past, just sample
• Trust-region algorithms for training responses: machine learning methods using indefinite Hessian approximations
• On High-Order Multilevel Optimization Strategies
• Subdivision-Based Nonlinear Multiscale Cloth Simulation
• Properties of a class of multilevel optimization algorithms for equality-constrained problems
• A Stochastic Approximation Method
• Adaptive multigrid methods for Signorini's problem in linear elasticity.

This page was built for publication: Globally Convergent Multilevel Training of Deep Residual Networks