A trust region method for noisy unconstrained optimization
From MaRDI portal
Abstract: Classical trust region methods were designed to solve problems in which function and gradient information are exact. This paper considers the case when there are bounded errors (or noise) in the above computations and proposes a simple modification of the trust region method to cope with these errors. The new algorithm only requires information about the size of the errors in the function evaluations and incurs no additional computational expense. It is shown that, when applied to a smooth (but not necessarily convex) objective function, the iterates of the algorithm visit a neighborhood of stationarity infinitely often, and that the rest of the sequence cannot stray too far away, as measured by function values. Numerical results illustrate how the classical trust region algorithm may fail in the presence of noise, and how the proposed algorithm ensures steady progress towards stationarity in these cases.
Recommendations
- A method of trust region type for minimizing noisy functions
- A retrospective trust-region method for unconstrained optimization
- Trust-region algorithms: probabilistic complexity and intrinsic noise with applications to subsampling techniques
- First- and second-order high probability complexity bounds for trust-region methods with noisy oracles
- An adaptive trust-region method without function evaluations
Cites work
- A stochastic line search method with expected complexity analysis
- A theoretical and empirical comparison of gradient approximations in derivative-free optimization
- Adaptive Finite-Difference Interval Estimation for Noisy Derivative-Free Optimization
- Adaptive sampling strategies for stochastic optimization
- Analysis of the BFGS Method with Errors
- Constrained Optimization in the Presence of Noise
- Convergence rate of incremental subgradient algorithms
- Derivative-Free Optimization of Noisy Functions via Quasi-Newton Methods
- Estimating Computational Noise
- Exact and inexact subsampled Newton methods for optimization
- Global Convergence Rate Analysis of a Generic Line Search Algorithm with Noise
- Global convergence rate analysis of unconstrained optimization methods based on probabilistic models
- scientific article; zbMATH DE number 4015993 (Why is no real title available?)
- Hybrid deterministic-stochastic methods for data fitting
- More test examples for nonlinear programming codes
- Multifidelity approaches for optimization under uncertainty
- Numerical experiments with the Lancelot package (Release \(A\)) for large-scale nonlinear optimization
- Numerical Optimization
- On sampling rates in simulation-based recursions
- On the Global Convergence of Trust Region Algorithms Using Inexact Gradient Information
- Optimization methods for large-scale machine learning
- Random gradient-free minimization of convex functions
- Sample size selection in optimization methods for machine learning
- Sequential quadratic optimization for nonlinear equality constrained stochastic optimization
- Stochastic optimization using a trust-region method and random models
- Survey of multifidelity methods in uncertainty propagation, inference, and optimization
- The impact of noise on evaluation complexity: the deterministic trust-region case
Cited in
(13)- Continuation methods with the trusty time-stepping scheme for linearly constrained optimization with noisy data
- A stochastic trust region method for unconstrained optimization problems
- Trust-region algorithms: probabilistic complexity and intrinsic noise with applications to subsampling techniques
- Convergence of Successive Linear Programming Algorithms for Noisy Functions
- Fully stochastic trust-region sequential quadratic programming for equality-constrained optimization problems
- A non-monotone trust-region method with noisy oracles and additional sampling
- First- and second-order high probability complexity bounds for trust-region methods with noisy oracles
- A sequential quadratic programming method with high-probability complexity bounds for nonlinear equality-constrained stochastic optimization
- Efficient proximal subproblem solvers for a nonsmooth trust-region method
- A line search framework with restarting for noisy optimization problems
- On some Moser-type iterative methods with applications to nonlinear problems
- Convergence analysis for a nonlocal gradient descent method via directional Gaussian smoothing
- A feasible method for constrained derivative-free optimization
This page was built for publication: A trust region method for noisy unconstrained optimization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6052069)
