A stability criterion for two timescale stochastic approximation schemes

DOI10.1016/j.automatica.2016.12.014zbMath1371.93208MaRDI QIDQ2409333

Shalabh Bhatnagar, Chandrashekar Lakshminarayanan

Publication date: 11 October 2017

Published in: Automatica (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1016/j.automatica.2016.12.014

zbMATH Keywords

simulation; reinforcement learning; two-timescale stochastic approximation; limiting ODE; stability of iterates

Mathematics Subject Classification ID

93C70: Time-scale analysis and singular perturbations in control/observation systems

93E12: Identification in stochastic control theory

93E35: Stochastic learning and adaptive control

Related Items

Convergence of stochastic approximation via martingale and converse Lyapunov methods, Sequential online subsampling for thinning experimental designs, Whittle index based Q-learning for restless bandits with average reward

Cites Work

• Unnamed Item
• Unnamed Item
• Natural actor-critic algorithms
• Stochastic approximation with two time scales
• Average cost temporal-difference learning
• Two-timescale simultaneous perturbation stochastic approximation using deterministic perturbation sequences
• Adaptive multivariate three-timescale stochastic approximation algorithms for simulation based optimization
• The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning
• A two Timescale Stochastic Approximation Scheme for Simulation-Based Parametric Optimization
• Perturbation theory and finite Markov chains