Hybrid deterministic-stochastic methods for data fitting
From MaRDI portal
Abstract: Many structured data-fitting applications require the solution of an optimization problem involving a sum over a potentially large number of measurements. Incremental gradient algorithms offer inexpensive iterations by sampling a subset of the terms in the sum. These methods can make great progress initially, but often slow as they approach a solution. In contrast, full-gradient methods achieve steady convergence at the expense of evaluating the full objective and gradient on each iteration. We explore hybrid methods that exhibit the benefits of both approaches. Rate-of-convergence analysis shows that by controlling the sample size in an incremental gradient algorithm, it is possible to maintain the steady convergence rates of full-gradient methods. We detail a practical quasi-Newton implementation based on this approach. Numerical experiments illustrate its potential benefits.
Recommendations
Cited in
(65)- A Levenberg-Marquardt method for large nonlinear least-squares problems with dynamic accuracy in functions and gradients
- On variance reduction for stochastic smooth convex optimization with multiplicative noise
- Sub-sampled Newton methods
- Variable sample size method for equality constrained optimization problems
- Stochastic relaxed inertial forward-backward-forward splitting for monotone inclusions in Hilbert spaces
- A frequency-domain analysis of inexact gradient methods
- Ritz-like values in steplength selections for stochastic gradient methods
- Subsampled nonmonotone spectral gradient methods
- Inexact first-order primal-dual algorithms
- Inexact restoration with subsampled trust-region methods for finite-sum minimization
- Modeling and parameter identification for a nonlinear multi-stage system for \textit{dha regulon} in batch culture
- Penalty variable sample size method for solving optimization problems with equality constraints in a form of mathematical expectation
- Nonmonotone line search methods with variable sample size
- Restricted strong convexity and its applications to convergence analysis of gradient-type methods in convex optimization
- Seismic iterative migration velocity analysis: two strategies to update the velocity model
- Robust optimal control for a batch nonlinear enzyme-catalytic switched time-delayed process with noisy output measurements
- Discriminative Bayesian filtering lends momentum to the stochastic Newton method for minimizing log-convex functions
- Inexact restoration approach for minimization with inexact evaluation of the objective function
- Erratum: ``Hybrid deterministic-stochastic methods for data fitting
- Block stochastic gradient iteration for convex and nonconvex optimization
- scientific article; zbMATH DE number 6982318 (Why is no real title available?)
- Adaptive sampling strategies for stochastic optimization
- Projected nonlinear least squares for exponential fitting
- On sampling rates in simulation-based recursions
- Automatic alignment for three-dimensional tomographic reconstruction
- Optimization methods for large-scale machine learning
- Total variation regularization strategies in full-waveform inversion
- A robust multi-batch L-BFGS method for machine learning
- A fully stochastic second-order trust region method
- Zeroth-order regularized optimization (ZORO): approximately sparse gradients and adaptive sampling
- A Variable Sample-Size Stochastic Quasi-Newton Method for Smooth and Nonsmooth Stochastic Convex Optimization
- On synchronous, asynchronous, and randomized best-response schemes for stochastic Nash games
- Minimizing finite sums with the stochastic average gradient
- A stochastic line search method with expected complexity analysis
- An inexact variable metric proximal point algorithm for generic quasi-Newton acceleration
- A stochastic semismooth Newton method for nonsmooth nonconvex optimization
- Newton-like method with diagonal correction for distributed optimization
- Convergence analysis of inexact randomized iterative methods
- An adaptive stochastic sequential quadratic programming with differentiable exact augmented Lagrangians
- Semi-discrete optimal transport: hardness, regularization and numerical solution
- Convergence analysis of a subsampled Levenberg-Marquardt algorithm
- Inequality constrained stochastic nonlinear optimization via active-set sequential quadratic programming
- A trust region method for noisy unconstrained optimization
- An adaptive sampling augmented Lagrangian method for stochastic optimization with deterministic constraints
- A framework of convergence analysis of mini-batch stochastic projected gradient methods
- Block mirror stochastic gradient method for stochastic optimization
- Hessian averaging in stochastic Newton methods achieves superlinear convergence
- A stochastic gradient method with variance control and variable learning rate for deep learning
- Estimating absorption and scattering in quantitative photoacoustic tomography with an adaptive Monte Carlo method for light transport
- Subsampled first-order optimization methods with applications in imaging
- AN-SPS: adaptive sample size nonmonotone line search spectral projected subgradient method for convex constrained optimization problems
- Spectral projected gradient method for stochastic optimization
- Stochastic ADMM with batch size adaptation for nonconvex nonsmooth optimization
- An adaptive importance sampling algorithm for risk-averse optimization
- Variable metric proximal stochastic gradient methods with additional sampling
- Convergence of ease-controlled random reshuffling gradient algorithms under Lipschitz smoothness
- Convergence analysis for a nonlocal gradient descent method via directional Gaussian smoothing
- Robust inversion, dimensionality reduction, and randomized sampling
- SLiSeS: subsampled line search spectral gradient method for finite sums
- Exploiting negative curvature in conjunction with adaptive sampling: theoretical results and a practical algorithm
- A survey of trust-region radius update mechanisms. Part I: First-order analysis
- Low-Rank Dependence Decomposition via Accelerated Symmetric Non-negative Matrix Factorization
- Multi-iteration stochastic optimizers
- A proximal Newton-type algorithm for zeroth-order stochastic composite optimization with a new norm test for sample size selection
- Generalized row-action methods for tomographic imaging
This page was built for publication: Hybrid deterministic-stochastic methods for data fitting
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2909270)
