Non-convex optimization for machine learning
From MaRDI portal
(Redirected from Publication:4643371)
Abstract: A vast majority of machine learning algorithms train their models and perform inference by solving optimization problems. In order to capture the learning and prediction problems accurately, structural constraints such as sparsity or low rank are frequently imposed or else the objective itself is designed to be a non-convex function. This is especially true of algorithms that operate in high-dimensional spaces or that train non-linear models such as tensor models and deep networks. The freedom to express the learning problem as a non-convex optimization problem gives immense modeling power to the algorithm designer, but often such problems are NP-hard to solve. A popular workaround to this has been to relax non-convex problems to convex ones and use traditional methods to solve the (convex) relaxed optimization problems. However this approach may be lossy and nevertheless presents significant challenges for large scale optimization. On the other hand, direct approaches to non-convex optimization have met with resounding success in several domains and remain the methods of choice for the practitioner, as they frequently outperform relaxation-based techniques - popular heuristics include projected gradient descent and alternating minimization. However, these are often poorly understood in terms of their convergence and other properties. This monograph presents a selection of recent advances that bridge a long-standing gap in our understanding of these heuristics. The monograph will lead the reader through several widely used non-convex optimization techniques, as well as applications thereof. The goal of this monograph is to both, introduce the rich literature in this area, as well as equip the reader with the tools and techniques needed to analyze these simple procedures for non-convex problems.
Recommendations
Cited in
(66)- Inertial proximal gradient methods with Bregman regularization for a class of nonconvex optimization problems
- On the geometric analysis of a quartic-quadratic optimization problem under a spherical constraint
- Proximal ADMM for nonconvex and nonsmooth optimization
- Provably training overparameterized neural network classifiers with non-convex constraints
- A unified Douglas-Rachford algorithm for generalized DC programming
- Finding the global optimum of a class of quartic minimization problem
- Machine learning algorithms of relaxation subgradient method with space extension
- A Bayesian perspective of statistical machine learning for big data
- Parametric deep energy approach for elasticity accounting for strain gradient effects
- A deep energy method for finite deformation hyperelasticity
- Stable and robust LQR design via scenario approach
- Nonsmooth rank-one matrix factorization landscape
- The exact worst-case convergence rate of the gradient method with fixed step lengths for \(L\)-smooth functions
- A backward SDE method for uncertainty quantification in deep learning
- Zeroth-order nonconvex stochastic optimization: handling constraints, high dimensionality, and saddle points
- A quasi-Newton approach to nonsmooth convex optimization problems in machine learning
- Systems of Bounded Rational Agents with Information-Theoretic Constraints
- A Newton-based method for nonconvex optimization with fast evasion of saddle points
- A finite time analysis of temporal difference learning with linear function approximation
- Why Do Local Methods Solve Nonconvex Problems?
- An inertial proximal alternating direction method of multipliers for nonconvex optimization
- A nonlinear matrix decomposition for mining the zeros of sparse data
- Low-rank, Orthogonally Decomposable Tensor Regression With Application to Visual Stimulus Decoding of fMRI Data
- Exact Recovery of Multichannel Sparse Blind Deconvolution via Gradient Descent
- Learning Enabled Constrained Black-Box Optimization
- Accelerating ill-conditioned low-rank matrix estimation via scaled gradient descent
- Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals
- Sublinear optimization for machine learning
- A combined dictionary learning and TV model for image restoration with convergence analysis
- Bilevel Methods for Image Reconstruction
- Orientation estimation of cryo-EM images using projected gradient descent method
- Nonlinear optimization and support vector machines
- Nonlinear optimization and support vector machines
- Sharp global convergence guarantees for iterative nonconvex optimization with random data
- CoolPINNs: a physics-informed neural network modeling of active cooling in vascular systems
- Optimal control under nonconvexity: A generalized Hamiltonian approach
- A unified analysis of stochastic gradient‐free Frank–Wolfe methods
- Nested alternating minimization with FISTA for non-convex and non-smooth optimization problems
- High-dimensional low-rank tensor autoregressive time series modeling
- An integrated design method for active fault diagnosis and control
- Tail probability estimates of continuous-time simulated annealing processes
- scientific article; zbMATH DE number 7709348 (Why is no real title available?)
- First-order methods for convex optimization
- Recent Theoretical Advances in Non-Convex Optimization
- Assessing Monotonicity: An Approach Based on Transformed Order Statistics
- Graphmax for text generation
- Joint learning of linear time-invariant dynamical systems
- On fluorophore imaging by nonlinear diffusion model with dynamical iterative scheme
- Optimization in machine learning: a distribution-space approach
- Extrapolated plug-and-play three-operator splitting methods for nonconvex optimization with applications to image restoration
- Exterior-point optimization for sparse and low-rank optimization
- Robust singular value decomposition with application to video surveillance background modelling
- Certified multifidelity zeroth-order optimization
- State space emulation and annealed sequential Monte Carlo for high dimensional optimization
- Using quadratic cuts to iteratively strengthen convexifications of box quadratic programs
- The minimizer of the sum of two strongly convex functions
- Bayesian penalized empirical likelihood and Markov chain Monte Carlo sampling
- Safe zeroth-order optimization using quadratic local approximations
- Analysis of optima set in a class of non-convex geometric optimization problems using bifurcation theory
- A physics-informed neural network model for the anisotropic hyperelasticity of the human passive myocardium
- Stochastic approach for price optimization problems with decision-dependent uncertainty
- Quantum Langevin dynamics for optimization
- A unified theoretical framework for the last-iterate convergence of stochastic adaptive optimization
- Minimization over the _p ball using a hybrid first-order method
- Title not available (Why is no real title available?)
- Measuring the local non-convexity of real algebraic curves
This page was built for publication: Non-convex optimization for machine learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4643371)
