Stochastic optimization using a trust-region method and random models
From MaRDI portal
Abstract: In this paper, we propose and analyze a trust-region model-based algorithm for solving unconstrained stochastic optimization problems. Our framework utilizes random models of an objective function , obtained from stochastic observations of the function or its gradient. Our method also utilizes estimates of function values to gauge progress that is being made. The convergence analysis relies on requirements that these models and these estimates are sufficiently accurate with sufficiently high, but fixed, probability. Beyond these conditions, no assumptions are made on how these models and estimates are generated. Under these general conditions we show an almost sure global convergence of the method to a first order stationary point. In the second part of the paper, we present examples of generating sufficiently accurate random models under biased or unbiased noise assumptions. Lastly, we present some computational results showing the benefits of the proposed method compared to existing approaches that are based on sample averaging or stochastic gradients.
Recommendations
- A stochastic trust region method for unconstrained optimization problems
- Stochastic derivative-free optimization using a trust region framework
- Stochastic trust-region methods with trust-region radius depending on probabilistic models
- Publication:4727236
- A quasi-Newton trust-region method for optimization under uncertainty using stochastic simplex approximate gradients
- scientific article; zbMATH DE number 5281595
- scientific article; zbMATH DE number 4055383
- scientific article; zbMATH DE number 1045614
- scientific article; zbMATH DE number 45868
- Stochastic variance reduced gradient methods using a trust-region-like scheme
Cites work
- A Stochastic Approximation Method
- Accelerated gradient methods for nonconvex nonlinear and stochastic programming
- Acceleration of Stochastic Approximation by Averaging
- Adaptive stochastic approximation by the simultaneous perturbation method
- Adaptive subgradient methods for online learning and stochastic optimization
- An optimal method for stochastic composite optimization
- Analysis of Sample-Path Optimization
- Benchmarking Derivative-Free Optimization Algorithms
- Convergence of trust-region methods based on probabilistic models
- Derivative-free optimization of expensive functions with computational error using weighted regression
- Global convergence of general derivative-free trust-region algorithms to first- and second-order critical points
- scientific article; zbMATH DE number 2121076 (Why is no real title available?)
- Introduction to Derivative-Free Optimization
- Introduction to Stochastic Search and Optimization
- Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function
- Multivariate stochastic approximation using a simultaneous perturbation gradient approximation
- On sampling rates in simulation-based recursions
- Optimization methods for large-scale machine learning
- Probability. Theory and examples.
- Robust Stochastic Approximation Approach to Stochastic Programming
- Stochastic derivative-free optimization using a trust region framework
- Stochastic Estimation of the Maximum of a Regression Function
- Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming
- The empirical behavior of sampling methods for stochastic programming
- UOBYQA: unconstrained optimization by quadratic approximation
- Variable-number sample-path optimization
Cited in
(79)- A Levenberg-Marquardt method for large nonlinear least-squares problems with dynamic accuracy in functions and gradients
- Global convergence rate analysis of unconstrained optimization methods based on probabilistic models
- Robust optimization of noisy blackbox problems using the mesh adaptive direct search algorithm
- Stochastic mesh adaptive direct search for blackbox optimization using probabilistic estimates
- Adaptive regularization for nonconvex optimization using inexact function values and randomly perturbed derivatives
- A zeroth order method for stochastic weakly convex optimization
- Expected complexity analysis of stochastic direct-search
- Linesearch Newton-CG methods for convex optimization with noise
- A stochastic first-order trust-region method with inexact restoration for finite-sum minimization
- A new nonmonotone adaptive trust region algorithm.
- Iteratively sampling scheme for stochastic optimization with variable number sample path
- Newton-type methods for non-convex optimization under inexact Hessian information
- A stochastic trust region method for unconstrained optimization problems
- Constrained stochastic blackbox optimization using a progressive barrier and probabilistic estimates
- The impact of noise on evaluation complexity: the deterministic trust-region case
- A trust-region algorithm with adaptive stochastic collocation for PDE optimization under uncertainty
- Convergence of trust-region methods based on probabilistic models
- Stochastic derivative-free optimization using a trust region framework
- scientific article; zbMATH DE number 4099039 (Why is no real title available?)
- ASTRO-DF: a class of adaptive sampling trust-region algorithms for derivative-free stochastic optimization
- A derivative-free trust-region algorithm for the optimization of functions smoothed via Gaussian convolution using adaptive multiple importance sampling
- Improving the flexibility and robustness of model-based derivative-free optimization solvers
- Stochastic analysis of an adaptive cubic regularization method under inexact gradient evaluations and dynamic Hessian accuracy
- A fully stochastic second-order trust region method
- A stochastic Levenberg-Marquardt method using random models with complexity results
- Stochastic trust-region methods with trust-region radius depending on probabilistic models
- Coupled learning enabled stochastic programming with endogenous uncertainty
- A stochastic trust-region framework for policy optimization
- Open problem: Iterative schemes for stochastic optimization: convergence statements and limit theorems
- Surrogate-based promising area search for Lipschitz continuous simulation optimization
- Convergence of Newton-MR under inexact Hessian information
- A stochastic line search method with expected complexity analysis
- Derivative-free optimization methods
- Solving Nonsmooth and Nonconvex Compound Stochastic Programs with Applications to Risk Measure Minimization
- SPT: A stochastic tunneling algorithm for global optimization
- Scalable subspace methods for derivative-free nonlinear least-squares optimization
- An adaptive stochastic sequential quadratic programming with differentiable exact augmented Lagrangians
- Convergence analysis of a subsampled Levenberg-Marquardt algorithm
- Inequality constrained stochastic nonlinear optimization via active-set sequential quadratic programming
- A trust region method for noisy unconstrained optimization
- An adaptive sampling augmented Lagrangian method for stochastic optimization with deterministic constraints
- TREGO: a trust-region framework for efficient global optimization
- Globally Convergent Multilevel Training of Deep Residual Networks
- Convergence Properties of an Objective-Function-Free Optimization Regularization Algorithm, Including an \(\boldsymbol{\mathcal{O}(\epsilon^{-3/2})}\) Complexity Bound
- Hessian averaging in stochastic Newton methods achieves superlinear convergence
- Trust-region algorithms: probabilistic complexity and intrinsic noise with applications to subsampling techniques
- Adaptive sampling quasi-Newton methods for zeroth-order stochastic optimization
- Riemannian Natural Gradient Methods
- Stochastic average model methods
- Fully stochastic trust-region sequential quadratic programming for equality-constrained optimization problems
- Stochastic trust-region and direct-search methods: a weak tail bound condition and reduced sample sizing
- Stochastic trust-region algorithm in random subspaces with convergence and expected complexity analyses
- Subsampled first-order optimization methods with applications in imaging
- A non-monotone trust-region method with noisy oracles and additional sampling
- First- and second-order high probability complexity bounds for trust-region methods with noisy oracles
- Worst case complexity bounds for linesearch-type derivative-free algorithms
- Derivative-free bound-constrained optimization for solving structured problems with surrogate models
- An investigation of stochastic trust-region based algorithms for finite-sum minimization
- A sequential quadratic programming method with high-probability complexity bounds for nonlinear equality-constrained stochastic optimization
- Complexity and performance for two classes of noise-tolerant first-order algorithms
- Variable metric proximal stochastic gradient methods with additional sampling
- Probabilistic iterative hard thresholding for sparse learning
- Model-and-search: a derivative-free local optimization algorithm
- Direct-search methods in the year 2025: theoretical guarantees and algorithmic paradigms
- Inexact Gauss-Newton methods with matrix approximation by sampling for nonlinear least-squares and systems
- Blackbox simulation optimization
- Complexity of zeroth- and first-order stochastic trust-region algorithms
- A randomized algorithm for nonconvex minimization with inexact evaluations and complexity guarantees
- A stochastic objective-function-free adaptive regularization method with optimal complexity
- The stochastic conjugate subgradient algorithm for kernel support vector machines
- Complexity bound of a Levenberg-Marquardt algorithm based on probabilistic Jacobian models
- Design guidelines for noise-tolerant optimization with applications in robust design
- Convergence analysis for a nonlocal gradient descent method via directional Gaussian smoothing
- A stochastic augmented Lagrangian method for stochastic convex programming
- Adaptive regularized quasi-Newton method using inexact first-order information
- Stochastic ISTA/FISTA adaptive step search algorithms for convex composite optimization
- A variable dimension sketching strategy for nonlinear least-squares
- A survey of trust-region radius update mechanisms. Part I: First-order analysis
- A discussion on variational analysis in derivative-free optimization
This page was built for publication: Stochastic optimization using a trust-region method and random models
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1646570)
