Incremental Majorization-Minimization Optimization with Application to Large-Scale Machine Learning

DOI10.1137/140957639zbMath1320.90047arXiv1402.4419MaRDI QIDQ5254990

Publication date: 11 June 2015

Published in: SIAM Journal on Optimization (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1402.4419

zbMATH Keywords

convex optimization; nonconvex optimization; majorization-minimization

Mathematics Subject Classification ID

90C25: Convex programming

90C06: Large-scale problems in mathematical programming

90C26: Nonconvex programming, global optimization

68T05: Learning and adaptive systems in artificial intelligence

Related Items

Unnamed Item, Unnamed Item, Unnamed Item, Composite Difference-Max Programs for Modern Statistical Estimation Problems, Global Convergence Rate of Proximal Incremental Aggregated Gradient Methods, Surpassing Gradient Descent Provably: A Cyclic Incremental Method with Linear Convergence Rate, A Coordinate-Descent Primal-Dual Algorithm with Large Step Size and Possibly Nonseparable Functions, Catalyst Acceleration for First-order Convex Optimization: from Theory to Practice, Unnamed Item, Proximal-Like Incremental Aggregated Gradient Method with Linear Convergence Under Bregman Distance Growth Conditions, Unnamed Item, Modulus-based iterative methods for constrained ℓ _p– ℓ _q minimization, Stochastic proximal quasi-Newton methods for non-convex composite optimization, Stochastic sub-sampled Newton method with variance reduction, An Inexact Variable Metric Proximal Point Algorithm for Generic Quasi-Newton Acceleration, Riemannian Stochastic Variance Reduced Gradient Algorithm with Retraction and Vector Transport, IQN: An Incremental Quasi-Newton Method with Local Superlinear Convergence Rate, A Bregman Forward-Backward Linesearch Algorithm for Nonconvex Composite Optimization: Superlinear Convergence to Nonisolated Local Minima, Incremental Quasi-Subgradient Method for Minimizing Sum of Geodesic Quasi-Convex Functions on Riemannian Manifolds with Applications, Bregman Finito/MISO for Nonconvex Regularized Finite Sum Minimization without Lipschitz Gradient Continuity, Stochastic Difference-of-Convex-Functions Algorithms for Nonconvex Programming, An aggressive reduction on the complexity of optimization for non-strongly convex objectives, Efficiency of higher-order algorithms for minimizing composite functions, Random-reshuffled SARAH does not need full gradient computations, Recent Theoretical Advances in Non-Convex Optimization, Stream-suitable optimization algorithms for some soft-margin support vector machine variants, The log-exponential smoothing technique and Nesterov's accelerated gradient method for generalized Sylvester problems, On the linear convergence of the approximate proximal splitting method for non-smooth convex optimization, Nonconvex nonsmooth optimization via convex-nonconvex majorization-minimization, Stochastic variance reduced gradient methods using a trust-region-like scheme, Incremental quasi-subgradient methods for minimizing the sum of quasi-convex functions, Generalized stochastic Frank-Wolfe algorithm with stochastic ``substitute gradient for structured convex optimization, Linear convergence of inexact descent method and inexact proximal gradient algorithms for lower-order regularization problems, Stochastic quasi-gradient methods: variance reduction via Jacobian sketching, An outer-inner linearization method for non-convex and nondifferentiable composite regularization problems, Stochastic DCA for minimizing a large sum of DC functions with application to multi-class logistic regression, A hybrid stochastic optimization framework for composite nonconvex optimization, Block-coordinate and incremental aggregated proximal gradient methods for nonsmooth nonconvex problems, A generalized proximal linearized algorithm for DC functions with application to the optimal size of the firm problem, Accelerating incremental gradient optimization with curvature information, Linear convergence of cyclic SAGA, Improved SVRG for finite sum structure optimization with application to binary classification, Optimizing cluster structures with inner product induced norm based dissimilarity measures: theoretical development and convergence analysis, Majorization-minimization generalized Krylov subspace methods for \({\ell _p}\)-\({\ell _q}\) optimization applied to image restoration, Proximal average approximated incremental gradient descent for composite penalty regularized empirical risk minimization, Convergence rates of accelerated proximal gradient algorithms under independent noise, Generalized forward-backward splitting with penalization for monotone inclusion problems, Coordinate descent with arbitrary sampling I: algorithms and complexity^†, Adaptive Sampling for Incremental Optimization Using Stochastic Gradient Descent

Uses Software

Cites Work