Convergence rate of linear two-time-scale stochastic approximation.
From MaRDI portal
Abstract: We study the rate of convergence of linear two-time-scale stochastic approximation methods. We consider two-time-scale linear iterations driven by i.i.d. noise, prove some results on their asymptotic covariance and establish asymptotic normality. The well-known result [Polyak, B. T. (1990). Automat. Remote Contr. 51 937-946; Ruppert, D. (1988). Technical Report 781, Cornell Univ.] on the optimality of Polyak-Ruppert averaging techniques specialized to linear stochastic approximation is established as a consequence of the general results in this paper.
Recommendations
- Convergence rate and averaging of nonlinear two-time-scale stochastic approximation algo\-rithms
- A generalization of the averaging procedure: the use of two-time-scale algorithms
- Stochastic Approximation with Averaging of the Iterates: Optimal Asymptotic Rate of Convergence for General Processes
- Two Time-Scale Stochastic Approximation with Controlled Markov Noise and Off-Policy Temporal-Difference Learning
- Convergence rates and decoupling in linear stochastic approximation algorithms
Cites work
- scientific article; zbMATH DE number 48727 (Why is no real title available?)
- scientific article; zbMATH DE number 3553601 (Why is no real title available?)
- scientific article; zbMATH DE number 976356 (Why is no real title available?)
- scientific article; zbMATH DE number 1043533 (Why is no real title available?)
- Acceleration of Stochastic Approximation by Averaging
- Actor-Critic--Type Learning Algorithms for Markov Decision Processes
- Applications of Singular Perturbation Techniques to Control Problems
- Convergence and convergence rate of iterative stochastic algorithms. I: General case
- New method of stochastic approximation type
- OnActor-Critic Algorithms
- Stochastic Approximation with Averaging of the Iterates: Optimal Asymptotic Rate of Convergence for General Processes
- Stochastic approximation methods for constrained and unconstrained systems
- Stochastic approximation with two time scales
Cited in
(33)- Fundamental design principles for reinforcement learning algorithms
- Non asymptotic controls on a recursive superquantile approximation
- Asymptotic behavior of multiscale stochastic partial differential equations with Hölder coefficients
- Networks of reinforced stochastic processes: asymptotics for the empirical means
- Stochastic approximation algorithms for superquantiles estimation
- Sequential online subsampling for thinning experimental designs
- Actor-critic algorithms with online feature adaptation
- A Two-Timescale Stochastic Algorithm Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Critic
- Generative adversarial networks are special cases of artificial curiosity (1990) and also closely related to predictability minimization (1991)
- Convergence rates and decoupling in linear stochastic approximation algorithms
- DIMIX: Diminishing Mixing for Sloppy Agents
- Online calibrated forecasts: memory efficiency versus universality for learning in games
- Empirical dynamic programming
- Computing VaR and CVaR using stochastic approximation and adaptive unconstrained importance sampling
- Weak convergence of dynamical systems in two timescales
- A stability criterion for two timescale stochastic approximation schemes
- Recursive regression estimation based on the two-time-scale stochastic approximation method and Bernstein polynomials
- Change-point monitoring for online stochastic approximations
- Gradient-based adaptive stochastic search for simulation optimization over continuous space
- Risk-Sensitive Reinforcement Learning via Policy Gradient Search
- A Two-Time-Scale Stochastic Optimization Framework with Applications in Control and Reinforcement Learning
- Towards multi‐agent reinforcement learning‐driven over‐the‐counter market simulations
- Two-time-scale nonparametric recursive regression estimator for independent functional data
- A generalization of the averaging procedure: the use of two-time-scale algorithms
- Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions
- Averaging principle and normal deviations for multiscale stochastic systems
- Finite-time analysis and restarting scheme for linear two-time-scale stochastic approximation
- Convergence rate and averaging of nonlinear two-time-scale stochastic approximation algo\-rithms
- Two-timescale stochastic gradient descent in continuous time with applications to joint online parameter estimation and optimal sensor placement
- Geometrical Insights for Implicit Generative Modeling
- Variance-constrained actor-critic algorithms for discounted and average reward MDPs
- Stochastic recursive inclusion in two timescales with an application to the Lagrangian dual problem
- GADE: a generative adversarial approach to density estimation and its applications
This page was built for publication: Convergence rate of linear two-time-scale stochastic approximation.
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1879892)
