Stochastic generalized gradient methods for training nonconvex nonsmooth neural networks

DOI10.1007/S10559-021-00397-ZzbMATH Open1494.65047OpenAlexW3204008412MaRDI QIDQ2058689FDOQ2058689

Vladimir I. Norkin

Publication date: 9 December 2021

Published in: Cybernetics and Systems Analysis (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1007/s10559-021-00397-z

zbMATH Keywords

deep learningmachine learningstochastic optimizationnonsmooth nonconvex optimizationmultilayer neural networksstochastic generalized gradient

Mathematics Subject Classification ID

Numerical mathematical programming methods (65K05) Artificial neural networks and deep learning (68T07) Nonconvex programming, global optimization (90C26) Stochastic programming (90C15)

Cites Work

• A Robust Gradient Sampling Algorithm for Nonsmooth, Nonconvex Optimization
• Learning representations by back-propagating errors
• A Stochastic Approximation Method
• Robust Stochastic Approximation Approach to Stochastic Programming
• Lectures on Stochastic Programming
• An Algorithm for Constrained Optimization with Semismooth Functions
• Stochastic generalized gradient method for nonconvex nonsmooth stochastic optimization
• Machine learning. The art and science of algorithms that make sense of data.
• Empirical risk minimization and problems of constructing linear classifiers
• Solution of nonconvex nonsmooth stochastic optimization problems
• Tame functions are semismooth
• Stochastic generalized-differentiable functions in the problem of nonconvex nonsmooth stochastic optimization
• Optimization Methods for Large-Scale Machine Learning
• Stochastic Model-Based Minimization of Weakly Convex Functions
• Generalized gradients in dynamic optimization, optimal control, and machine learning problems
• Stochastic subgradient method converges on tame functions
• Step control for direct stochastic-programming methods
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)

Cited In (4)

• Title not available (Why is that?)
• Subgradient-based feedback neural networks for non-differentiable convex optimization problems
• Subgradient Sampling for Nonsmooth Nonconvex Minimization
• Taming Neural Networks with TUSLA: Nonconvex Learning via Adaptive Stochastic Gradient Langevin Algorithms

Recommendations

• Title not available (Why is that?) 👍 👎
• Stochastic generalized gradient method for nonconvex nonsmooth stochastic optimization 👍 👎
• Generalized Neural Network for Nonsmooth Nonlinear Programming Problems 👍 👎
• Non-convergence of stochastic gradient descent in the training of deep neural networks 👍 👎
• Neural network for nonsmooth pseudoconvex optimization with general convex constraints 👍 👎
• The convergence of stochastic gradient algorithms applied to learning in neural networks 👍 👎
• Neural network smoothing approximation method for stochastic variational inequality problems 👍 👎
• SOM and Neural Gas as Graduated Nonconvexity Algorithms 👍 👎
• Nonlinear Gradient Mappings and Stochastic Optimization: A General Framework with Applications to Heavy-Tail Noise 👍 👎
• Stochastic perturbation of subgradient algorithm for nonconvex deep neural networks 👍 👎

This page was built for publication: Stochastic generalized gradient methods for training nonconvex nonsmooth neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2058689)