An inexact variable metric proximal point algorithm for generic quasi-Newton acceleration
From MaRDI portal
(Redirected from Publication:5231671)
Abstract: We propose an inexact variable-metric proximal point algorithm to accelerate gradient-based optimization algorithms. The proposed scheme, called QNing can be notably applied to incremental first-order methods such as the stochastic variance-reduced gradient descent algorithm (SVRG) and other randomized incremental optimization algorithms. QNing is also compatible with composite objectives, meaning that it has the ability to provide exactly sparse solutions when the objective involves a sparsity-inducing regularization. When combined with limited-memory BFGS rules, QNing is particularly effective to solve high-dimensional optimization problems, while enjoying a worst-case linear convergence rate for strongly convex problems. We present experimental results where QNing gives significant improvements over competing methods for training machine learning methods on large samples and in high dimensions.
Recommendations
- Proximal quasi-Newton methods for regularized convex optimization with linear and accelerated sublinear convergence rates
- On quasi-Newton forward-backward splitting: proximal calculus and convergence
- Practical inexact proximal quasi-Newton method with global complexity analysis
- IQN: an incremental quasi-Newton method with local superlinear convergence rate
- A Variable Sample-Size Stochastic Quasi-Newton Method for Smooth and Nonsmooth Stochastic Convex Optimization
Cites work
- A Characterization of Superlinear Convergence and Its Application to Quasi-Newton Methods
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
- A Globally and Superlinearly Convergent Algorithm for Nonsmooth Convex Minimization
- A proximal stochastic gradient method with progressive variance reduction
- A quasi-second-order proximal bundle algorithm
- A stochastic quasi-Newton method for large-scale optimization
- A unified convergence analysis of block successive minimization methods for nonsmooth optimization
- Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization
- An inexact successive quadratic approximation method for L-1 regularized optimization
- Catalyst acceleration for first-order convex optimization: from theory to practice
- Convex optimization algorithms
- Descentwise inexact proximal algorithms for smooth optimization
- Efficiency of coordinate descent methods on huge-scale optimization problems
- Forward-backward quasi-Newton methods for nonsmooth optimization problems
- Global Convergence of a Cass of Quasi-Newton Methods on Convex Problems
- Global convergence of online limited memory BFGS
- Global convergence rate analysis of unconstrained optimization methods based on probabilistic models
- Gradient methods for minimizing composite functions
- scientific article; zbMATH DE number 3850830 (Why is no real title available?)
- scientific article; zbMATH DE number 3678487 (Why is no real title available?)
- scientific article; zbMATH DE number 5060482 (Why is no real title available?)
- scientific article; zbMATH DE number 3192366 (Why is no real title available?)
- Hybrid deterministic-stochastic methods for data fitting
- Incremental majorization-minimization optimization with application to large-scale machine learning
- Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function
- Minimizing finite sums with the stochastic average gradient
- Monotone Operators and the Proximal Point Algorithm
- New Proximal Point Algorithms for Convex Minimization
- Numerical optimization. Theoretical and practical aspects. Transl. from the French
- On the limited memory BFGS method for large scale optimization
- On the superlinear convergence of the variable metric proximal point algorithm using Broyden and BFGS matrix secant updating
- Practical Aspects of the Moreau--Yosida Regularization: Theoretical Preliminaries
- Practical inexact proximal quasi-Newton method with global complexity analysis
- Proximal quasi-Newton methods for nondifferentiable convex optimization
- Proximal quasi-Newton methods for regularized convex optimization with linear and accelerated sublinear convergence rates
- Randomized smoothing for stochastic optimization
- Regularization and Variable Selection Via the Elastic Net
- Sample size selection in optimization methods for machine learning
- Smooth minimization of non-smooth functions
- Smoothing and first order methods: a unified framework
- Sparse and redundant representations. From theory to applications in signal and image processing.
- Sparse modeling for image and vision processing
- Updating Quasi-Newton Matrices with Limited Storage
Cited in
(8)- Proximal quasi-Newton methods for regularized convex optimization with linear and accelerated sublinear convergence rates
- An inexact successive quadratic approximation method for a class of difference-of-convex optimization problems
- The developments of proximal point algorithms
- A modified conjugate gradient method for general convex functions
- Practical inexact proximal quasi-Newton method with global complexity analysis
- Distributed Learning with Sparse Communications by Identification
- On quasi-Newton forward-backward splitting: proximal calculus and convergence
- IQN: an incremental quasi-Newton method with local superlinear convergence rate
This page was built for publication: An inexact variable metric proximal point algorithm for generic quasi-Newton acceleration
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5231671)
