Non-convex optimization for machine learning
From MaRDI portal
Publication:4643371
Abstract: A vast majority of machine learning algorithms train their models and perform inference by solving optimization problems. In order to capture the learning and prediction problems accurately, structural constraints such as sparsity or low rank are frequently imposed or else the objective itself is designed to be a non-convex function. This is especially true of algorithms that operate in high-dimensional spaces or that train non-linear models such as tensor models and deep networks. The freedom to express the learning problem as a non-convex optimization problem gives immense modeling power to the algorithm designer, but often such problems are NP-hard to solve. A popular workaround to this has been to relax non-convex problems to convex ones and use traditional methods to solve the (convex) relaxed optimization problems. However this approach may be lossy and nevertheless presents significant challenges for large scale optimization. On the other hand, direct approaches to non-convex optimization have met with resounding success in several domains and remain the methods of choice for the practitioner, as they frequently outperform relaxation-based techniques - popular heuristics include projected gradient descent and alternating minimization. However, these are often poorly understood in terms of their convergence and other properties. This monograph presents a selection of recent advances that bridge a long-standing gap in our understanding of these heuristics. The monograph will lead the reader through several widely used non-convex optimization techniques, as well as applications thereof. The goal of this monograph is to both, introduce the rich literature in this area, as well as equip the reader with the tools and techniques needed to analyze these simple procedures for non-convex problems.
Recommendations
Cited in
(54)- A deep energy method for finite deformation hyperelasticity
- High-dimensional low-rank tensor autoregressive time series modeling
- An integrated design method for active fault diagnosis and control
- Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals
- A quasi-Newton approach to nonsmooth convex optimization problems in machine learning
- A unified Douglas-Rachford algorithm for generalized DC programming
- Finding the global optimum of a class of quartic minimization problem
- A unified analysis of stochastic gradient‐free Frank–Wolfe methods
- Machine learning algorithms of relaxation subgradient method with space extension
- Stable and robust LQR design via scenario approach
- Assessing Monotonicity: An Approach Based on Transformed Order Statistics
- A nonlinear matrix decomposition for mining the zeros of sparse data
- On the geometric analysis of a quartic-quadratic optimization problem under a spherical constraint
- Joint learning of linear time-invariant dynamical systems
- Measuring the local non-convexity of real algebraic curves
- Learning Enabled Constrained Black-Box Optimization
- A combined dictionary learning and TV model for image restoration with convergence analysis
- scientific article; zbMATH DE number 7709348 (Why is no real title available?)
- Zeroth-order nonconvex stochastic optimization: handling constraints, high dimensionality, and saddle points
- Inertial proximal gradient methods with Bregman regularization for a class of nonconvex optimization problems
- Sublinear optimization for machine learning
- Bilevel Methods for Image Reconstruction
- Optimization in machine learning: a distribution-space approach
- Low-rank, Orthogonally Decomposable Tensor Regression With Application to Visual Stimulus Decoding of fMRI Data
- Exact Recovery of Multichannel Sparse Blind Deconvolution via Gradient Descent
- Optimal control under nonconvexity: A generalized Hamiltonian approach
- Graphmax for text generation
- A Newton-based method for nonconvex optimization with fast evasion of saddle points
- Nonlinear optimization and support vector machines
- Proximal ADMM for nonconvex and nonsmooth optimization
- Tail probability estimates of continuous-time simulated annealing processes
- An inertial proximal alternating direction method of multipliers for nonconvex optimization
- Nonlinear optimization and support vector machines
- Systems of Bounded Rational Agents with Information-Theoretic Constraints
- First-order methods for convex optimization
- Recent Theoretical Advances in Non-Convex Optimization
- Orientation estimation of cryo-EM images using projected gradient descent method
- scientific article; zbMATH DE number 7415093 (Why is no real title available?)
- Provably training overparameterized neural network classifiers with non-convex constraints
- A finite time analysis of temporal difference learning with linear function approximation
- On fluorophore imaging by nonlinear diffusion model with dynamical iterative scheme
- Nonsmooth rank-one matrix factorization landscape
- The exact worst-case convergence rate of the gradient method with fixed step lengths for \(L\)-smooth functions
- A backward SDE method for uncertainty quantification in deep learning
- Nested alternating minimization with FISTA for non-convex and non-smooth optimization problems
- Parametric deep energy approach for elasticity accounting for strain gradient effects
- A Bayesian perspective of statistical machine learning for big data
- Sharp global convergence guarantees for iterative nonconvex optimization with random data
- CoolPINNs: a physics-informed neural network modeling of active cooling in vascular systems
- Extrapolated plug-and-play three-operator splitting methods for nonconvex optimization with applications to image restoration
- State space emulation and annealed sequential Monte Carlo for high dimensional optimization
- Certified multifidelity zeroth-order optimization
- Exterior-point optimization for sparse and low-rank optimization
- Robust singular value decomposition with application to video surveillance background modelling
This page was built for publication: Non-convex optimization for machine learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4643371)
