Recursive Stochastic Algorithms for Global Optimization in \mathbb{R}^d
From MaRDI portal
Publication:3978271
This paper deals with a class of algorithms for finding a global minimum of a smooth function \(U(x)\), \(x\in \mathbb{R}^ d\). More precisely the authors analyze the convergence of a modified stochastic gradient algorithm: \(X_{k+1}=X_ k-a_ k(\nabla U(X_ k)+\xi_ k)+b_ kW_ k\), where \(\xi_ k\) is a sequence of \(\mathbb{R}^ d\) valued random variables, \(W_ k\) is a sequence of standard \(d\)-dimensional independent Gaussian random variables, and \(a_ k\), \(b_ k\) are sequences of positive numbers with \(a_ k\), \(b_ k\to 0\). This algorithm is obtained by adding the \(b_ kW_ k\) term to a standard stochastic gradient algorithm. The ideal behind the additional \(b_ kW_ k\) term is that if \(b_ k\) tends to zero slowly enough then possibly \(X_ k\) will avoid getting trapped in a strictly local minimum of \(U\). Conditions and relations on the different parameters are given for ensuring convergence in probability towards a global minimum. No evaluation of the convergence speed is given but the paper is interesting.
Recommendations
- scientific article; zbMATH DE number 89067
- Stochastic recursive algorithms for optimization. Simultaneous perturbation methods
- Stochastic global maximum principle for optimization with recursive utilities
- Stochastic optimization of forward recursive functions
- Stochastic Methods for Global Optimization
- Stochastic methods for practical global optimization
- Global Random Optimization by Simultaneous Perturbation Stochastic Approximation
- A stochastic algorithm for constrained global optimization
Cited in
(63)- A strong approximation theorem for stochastic recursive algorithms
- Weak convergence rates for stochastic approximation with application to multiple targets and simulated annealing
- Recursive identification in continuous-time stochastic processes
- Global optimization by random perturbation of the gradient method with a fixed parameter
- Stochastic approximation of global minimum points
- Convergence of the simulated annealing algorithm for continuous global optimization
- Almost surely convergent global optimziation algorithm using noise-corrupted observations
- Stability of annealing schemes and related processes
- Distributed stochastic algorithm for global optimization in networked system
- Stochastic optimization of forward recursive functions
- A stochastic optimization method for the evaluation of minima up to
- An adaptive simulated annealing algorithm.
- Simulated annealing algorithms for continuous global optimization: Convergence conditions
- Simultaneous small noise limit for singularly perturbed slow-fast coupled diffusions
- Stochastic gradient Hamiltonian Monte Carlo for non-convex learning
- Non-asymptotic guarantees for sampling by stochastic gradient descent
- Stochastic global maximum principle for optimization with recursive utilities
- Improving simulated annealing through derandomization
- Convergence rates of Gibbs measures with degenerate minimum
- Optimising portfolio diversification and dimensionality
- Multiscale Q-learning with linear function approximation
- APPROACHES TO ADAPTIVE STOCHASTIC SEARCH BASED ON THE NONEXTENSIVE q-DISTRIBUTION
- A Conditional Gaussian Martingale Algorithm for Global Optimization
- Convergence of a global stochastic optimization algorithm with partial step size restarting
- Numerical optimization and quasiconvexity
- scientific article; zbMATH DE number 899666 (Why is no real title available?)
- scientific article; zbMATH DE number 926779 (Why is no real title available?)
- Langevin dynamics for adaptive inverse reinforcement learning of stochastic gradient algorithms
- Convergence of Recursive Stochastic Algorithms Using Wasserstein Divergence
- One-dimensional system arising in stochastic gradient descent
- Some limit properties of Markov chains induced by recursive stochastic algorithms
- Global convergence of stochastic gradient Hamiltonian Monte Carlo for nonconvex stochastic optimization: nonasymptotic performance bounds and momentum-based acceleration
- State-Dependent Temperature Control for Langevin Diffusions
- A concentration bound for stochastic approximation via Alekseev's formula
- Second-order guarantees of distributed gradient algorithms
- Replica exchange for non-convex optimization
- A proof of convergence of general stochastic search for global minimum
- Multiscale Stochastic Approximation for Parametric Optimization of Hidden Markov Models
- Embedding Learning
- Stochastic approximation
- Approximation of an analog diffusion network with applications to image estimation
- Managing interprocessor delays in distributed recursive algorithms
- Stochastic approximation algorithms: overview and recent trends.
- Unadjusted Langevin algorithm with multiplicative noise: total variation and Wasserstein bounds
- On the Generalized Langevin Equation for Simulated Annealing
- Swarm gradient dynamics for global optimization: the mean-field limit case
- Delay-induced homoclinic bifurcations in modified gradient bistable systems and their relevance to optimization
- Optimization with delay-induced bifurcations
- Adaptive state-dependent diffusion for derivative-free optimization
- Swing contract pricing: with and without neural networks
- Riemannian Langevin algorithm for solving semidefinite programs
- Discrete-time simulated annealing: a convergence analysis via the Eyring-Kramers law
- Convergence error analysis of reflected gradient Langevin dynamics for non-convex constrained optimization
- Simulated annealing simulated
- Stochastic approximation with two time scales: the general case
- Convergence rates for inhomogeneous Markov chains from stochastic approximation algorithms
- Stochastic asymptotical regularization for nonlinear ill-posed problems
- Gradient is all you need? How consensus-based optimization can be interpreted as a stochastic relaxation of gradient descent
- Error estimates between SGD with momentum and underdamped Langevin diffusion
- Data-driven stochastic asymptotic regularization for ill-posed inverse problems with a posteriori and heuristic stopping
- Convergence analysis of a global optimization algorithm using stochastic differential equations
- Linearly constrained global optimization and stochastic differential equations
- Global optimization using diffusion perturbations with large noise intensity
This page was built for publication: Recursive Stochastic Algorithms for Global Optimization in $\mathbb{R}^d $
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3978271)
