Convergence of trust-region methods based on probabilistic models
From MaRDI portal
Abstract: In this paper we consider the use of probabilistic or random models within a classical trust-region framework for optimization of deterministic smooth general nonlinear functions. Our method and setting differs from many stochastic optimization approaches in two principal ways. Firstly, we assume that the value of the function itself can be computed without noise, in other words, that the function is deterministic. Secondly, we use random models of higher quality than those produced by usual stochastic gradient methods. In particular, a first order model based on random approximation of the gradient is required to provide sufficient quality of approximation with probability greater than or equal to 1/2. This is in contrast with stochastic gradient approaches, where the model is assumed to be "correct" only in expectation. As a result of this particular setting, we are able to prove convergence, with probability one, of a trust-region method which is almost identical to the classical method. Hence we show that a standard optimization framework can be used in cases when models are random and may or may not provide good approximations, as long as "good" models are more likely than "bad" models. Our results are based on the use of properties of martingales. Our motivation comes from using random sample sets and interpolation models in derivative-free optimization. However, our framework is general and can be applied with any source of uncertainty in the model. We discuss various applications for our methods in the paper.
Recommendations
- Stochastic optimization using a trust-region method and random models
- Complexity and global rates of trust-region methods based on probabilistic models
- Global convergence rate analysis of unconstrained optimization methods based on probabilistic models
- Global convergence of general derivative-free trust-region algorithms to first- and second-order critical points
- Stochastic trust-region methods with trust-region radius depending on probabilistic models
Cited in
(71)- A Levenberg-Marquardt method for large nonlinear least-squares problems with dynamic accuracy in functions and gradients
- A progressive barrier derivative-free trust-region algorithm for constrained optimization
- Global convergence rate analysis of unconstrained optimization methods based on probabilistic models
- Stochastic optimization using a trust-region method and random models
- Stochastic mesh adaptive direct search for blackbox optimization using probabilistic estimates
- Expected complexity analysis of stochastic direct-search
- Levenberg-Marquardt method based on probabilistic Jacobian models for nonlinear equations
- Linesearch Newton-CG methods for convex optimization with noise
- A stochastic first-order trust-region method with inexact restoration for finite-sum minimization
- Efficient unconstrained black box optimization
- Newton-type methods for non-convex optimization under inexact Hessian information
- Projected adaptive cubic regularization algorithm with derivative-free filter technique for box constrained optimization
- Recent advances in trust region algorithms
- Direct search based on probabilistic feasible descent for bound and linearly constrained problems
- Constrained stochastic blackbox optimization using a progressive barrier and probabilistic estimates
- Bound-constrained global optimization of functions with low effective dimensionality using multiple random embeddings
- The impact of noise on evaluation complexity: the deterministic trust-region case
- Stochastic derivative-free optimization using a trust region framework
- Levenberg-Marquardt methods based on probabilistic gradient models and inexact subproblem solution, with application to data assimilation
- Controlling model trust with compactly supported smooth RBF
- Descent direction method with line search for unconstrained optimization in noisy environment
- scientific article; zbMATH DE number 726885 (Why is no real title available?)
- Complexity and global rates of trust-region methods based on probabilistic models
- ASTRO-DF: a class of adaptive sampling trust-region algorithms for derivative-free stochastic optimization
- A derivative-free trust-region algorithm for the optimization of functions smoothed via Gaussian convolution using adaptive multiple importance sampling
- A derivative-free affine scaling trust region methods based on probabilistic models with new nonmonotone line search technique for linear inequality constrained minimization without strict complementarity
- A fully stochastic second-order trust region method
- A stochastic Levenberg-Marquardt method using random models with complexity results
- Stochastic trust-region methods with trust-region radius depending on probabilistic models
- Global Linear Convergence of Evolution Strategies on More than Smooth Strongly Convex Functions
- Coupled learning enabled stochastic programming with endogenous uncertainty
- A stochastic trust-region framework for policy optimization
- Convergence of Newton-MR under inexact Hessian information
- Trust-region methods for the derivative-free optimization of nonsmooth black-box functions
- A stochastic line search method with expected complexity analysis
- Derivative-free optimization methods
- Adaptive Regularization Algorithms with Inexact Evaluations for Nonconvex Optimization
- Direct search based on probabilistic descent
- Scalable subspace methods for derivative-free nonlinear least-squares optimization
- An adaptive stochastic sequential quadratic programming with differentiable exact augmented Lagrangians
- Convergence analysis of a subsampled Levenberg-Marquardt algorithm
- Inequality constrained stochastic nonlinear optimization via active-set sequential quadratic programming
- Direct Search Based on Probabilistic Descent in Reduced Spaces
- Optimization by moving ridge functions: derivative-free optimization for computationally intensive functions
- Global optimization using random embeddings
- Trust-region algorithms: probabilistic complexity and intrinsic noise with applications to subsampling techniques
- Stochastic average model methods
- Stochastic trust-region and direct-search methods: a weak tail bound condition and reduced sample sizing
- Convergence and worst-case complexity of adaptive Riemannian trust-region methods for optimization on manifolds
- First- and second-order high probability complexity bounds for trust-region methods with noisy oracles
- On the global complexity of a derivative-free Levenberg-Marquardt algorithm via orthogonal spherical smoothing
- Derivative-free bound-constrained optimization for solving structured problems with surrogate models
- An investigation of stochastic trust-region based algorithms for finite-sum minimization
- A sequential quadratic programming method with high-probability complexity bounds for nonlinear equality-constrained stochastic optimization
- SketchySGD: reliable stochastic optimization via randomized curvature estimates
- Least H^2 norm updating of quadratic interpolation models for derivative-free trust-region algorithms
- Variable metric proximal stochastic gradient methods with additional sampling
- Probabilistic iterative hard thresholding for sparse learning
- An improved randomized algorithm with noise level tuning for large-scale noisy unconstrained DFO problems
- Derivative-free optimization with transformed objective functions and the algorithm based on the least Frobenius norm updating quadratic model
- A derivative-free geometric algorithm for optimization on a sphere
- A multi-precision quadratic regularization method for unconstrained optimization with rounding error analysis
- Optimization problems governed by systems of PDEs with uncertainties
- The stochastic conjugate subgradient algorithm for kernel support vector machines
- Complexity bound of a Levenberg-Marquardt algorithm based on probabilistic Jacobian models
- Convergence analysis for a nonlocal gradient descent method via directional Gaussian smoothing
- Stochastic ISTA/FISTA adaptive step search algorithms for convex composite optimization
- A survey of trust-region radius update mechanisms. Part I: First-order analysis
- A derivative-free Levenberg-Marquardt method for sparse nonlinear least squares problems
- A concise training scheme of RBF neural networks with fixed center points for partial differential equations with Dirichlet boundary conditions
- Survey of derivative-free optimization
This page was built for publication: Convergence of trust-region methods based on probabilistic models
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2934477)
