A stability criterion for two timescale stochastic approximation schemes
From MaRDI portal
Publication:2409333
DOI10.1016/j.automatica.2016.12.014zbMath1371.93208MaRDI QIDQ2409333
Shalabh Bhatnagar, Chandrashekar Lakshminarayanan
Publication date: 11 October 2017
Published in: Automatica (Search for Journal in Brave)
Full work available at URL: https://doi.org/10.1016/j.automatica.2016.12.014
simulation; reinforcement learning; two-timescale stochastic approximation; limiting ODE; stability of iterates
93C70: Time-scale analysis and singular perturbations in control/observation systems
93E12: Identification in stochastic control theory
93E35: Stochastic learning and adaptive control
Related Items
Convergence of stochastic approximation via martingale and converse Lyapunov methods, Sequential online subsampling for thinning experimental designs, Whittle index based Q-learning for restless bandits with average reward
Cites Work
- Unnamed Item
- Unnamed Item
- Natural actor-critic algorithms
- Stochastic approximation with two time scales
- Average cost temporal-difference learning
- Two-timescale simultaneous perturbation stochastic approximation using deterministic perturbation sequences
- Adaptive multivariate three-timescale stochastic approximation algorithms for simulation based optimization
- The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning
- A two Timescale Stochastic Approximation Scheme for Simulation-Based Parametric Optimization
- Perturbation theory and finite Markov chains