Nonmonotone line search methods with variable sample size
The paper deals with nonmonotone line search methods for unconstrained optimization. The objective function has the form of mathematical expectation and is approximated by the sample average approximation (SAA) with a large sample of a fixed size. As the function evaluation is expensive, general methods that start with a small sample and increase the sample size throughout the optimization process are usually considered. The aim is to ensure increasing precision during the optimization procedure regardless of the behavior of the objective function. In this paper, the authors introduce and analyze a class of algorithms that combines nonmonotone line search rules with a variable sample size strategy and extend the results developed by \textit{N. Krejić} and \textit{N. Krklec} [J. Comput. Appl. Math. 245, 213--231 (2013; Zbl 1262.65066)]. The sample size may oscillate in each iteration in accordance with the progress made in a decrease of the objective function and the precision measured by an approximate width of the confidence interval. The proposed methods result in approximate solutions for the SAA problem with significantly smaller computational costs than the classical SAA method. A complete algorithm is presented and global convergence results for general search directions are proven. The R-linear rate of convergence is obtained when the gradient of objective function is available and a descent search direction is used at every iteration. Extensive numerical experiments on academic optimization problems in noisy environment as well as on problems with real data are carried out.
- A nonmonotone line search method for stochastic optimization problems
- Line search methods with variable sample size for unconstrained optimization
- A nonmonotone line search method for noisy minimization
- Variable-sample methods for stochastic optimization
- Nonmonotone line searches for optimization algorithms
- A class on nonmonotone stabilization methods in unconstrained optimization
- A derivative-free line search and global convergence of Broyden-like method for nonlinear equations
- A derivative-free nonmonotone line search and its application to the spectral residual method
- A derivative-free nonmonotone line-search technique for unconstrained optimization
- A Nonmonotone Line Search Technique and Its Application to Unconstrained Optimization
- A Nonmonotone Line Search Technique for Newton’s Method
- A nonmonotone spectral projected gradient method for large-scale topology optimization problems
- An adaptive Monte Carlo algorithm for computing mixed logit estimators
- An Assessment of Nonmonotone Linesearch Techniques for Unconstrained Optimization
- An experimental methodology for response surface optimization methods
- Benchmarking optimization software with performance profiles.
- Convergence theory for nonconvex stochastic programming with an application to mixed logit
- Efficient sample sizes in stochastic nonlinear programming
- Globally convergent inexact quasi-Newton methods for solving nonlinear systems
- Globally convergent Jacobian smoothing inexact Newton methods for NCP
- Hybrid deterministic-stochastic methods for data fitting
- Introduction to Stochastic Search and Optimization
- Introductory lectures on convex optimization. A basic course.
- Line search methods with variable sample size for unconstrained optimization
- Numerical Optimization
- On Choosing Parameters in Retrospective-Approximation Algorithms for Stochastic Root Finding and Simulation Optimization
- On the nonmonotone line search
- On the use of stochastic Hessian information in optimization methods for machine learning
- Optimality functions in stochastic programming
- Sample size selection in optimization methods for machine learning
- Spectral residual method without gradient information for solving large-scale nonlinear systems of equations
- The Barzilai and Borwein Gradient Method for the Large Scale Unconstrained Minimization Problem
- Variable-number sample-path optimization
- Variable-sample methods for stochastic optimization
- A Levenberg-Marquardt method for large nonlinear least-squares problems with dynamic accuracy in functions and gradients
- Variable sample size method for equality constrained optimization problems
- Line search methods with variable sample size for unconstrained optimization
- Tensor Bernstein concentration inequalities with an application to sample estimators for high-order moments
- Subsampled nonmonotone spectral gradient methods
- Inexact restoration with subsampled trust-region methods for finite-sum minimization
- Penalty variable sample size method for solving optimization problems with equality constraints in a form of mathematical expectation
- Barzilai–Borwein method with variable sample size for stochastic linear complementarity problems
- Inexact restoration approach for minimization with inexact evaluation of the objective function
- Variable-sample methods for stochastic optimization
- On the employment of inexact restoration for the minimization of functions whose evaluation is subject to errors
- Greedy Sampling Using Nonlinear Optimization
- A nonmonotone line search method for noisy minimization
- A nonmonotone line search method for stochastic optimization problems
- Newton-like method with diagonal correction for distributed optimization
- A stochastic gradient method with variance control and variable learning rate for deep learning
- Subsampled first-order optimization methods with applications in imaging
- A non-monotone trust-region method with noisy oracles and additional sampling
- AN-SPS: adaptive sample size nonmonotone line search spectral projected subgradient method for convex constrained optimization problems
- Spectral projected gradient method for stochastic optimization
- Spectral stochastic gradient method with additional sampling for finite and infinite sums
- A variable metric proximal stochastic gradient method: an application to classification problems
This page was built for publication: Nonmonotone line search methods with variable sample size
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2340358)
