A stability criterion for two timescale stochastic approximation schemes

DOI10.1016/J.AUTOMATICA.2016.12.014MaRDI QIDQ2409333zbMATH OpenOpenAlexFDO

Authors Chandrashekar Lakshminarayanan, Shalabh Bhatnagar

Publication date 11 October 2017

Published in Automatica (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1016/j.automatica.2016.12.014

zbMATH Keywords

limiting ODEreinforcement learningsimulationstability of iteratestwo-timescale stochastic approximation

Mathematics Subject Classification ID

Time-scale analysis and singular perturbations in control/observation systems (93C70) Identification in stochastic control theory (93E12) Stochastic learning and adaptive control (93E35)

Recommendations

• scientific article; zbMATH DE number 1368826
• Stochastic approximation with two time scales
• Nonlinear Two-Time-Scale Stochastic Approximation: Convergence and Finite-Time Performance
• Asymptotic mean-square stability of two-step methods for stochastic ordinary differential equations
• scientific article; zbMATH DE number 834475
• scientific article; zbMATH DE number 2186195
• scientific article; zbMATH DE number 474424
• scientific article; zbMATH DE number 12731
• Convergence rate of linear two-time-scale stochastic approximation.

Cites work

• A two Timescale Stochastic Approximation Scheme for Simulation-Based Parametric Optimization
• Adaptive multivariate three-timescale stochastic approximation algorithms for simulation based optimization
• Average cost temporal-difference learning
• scientific article; zbMATH DE number 5348356 (Why is no real title available?)
• scientific article; zbMATH DE number 1321699 (Why is no real title available?)
• Natural actor-critic algorithms
• Perturbation theory and finite Markov chains
• Stochastic approximation with two time scales
• The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning
• Two-timescale simultaneous perturbation stochastic approximation using deterministic perturbation sequences

Cited in

(13)

• Whittle index based Q-learning for restless bandits with average reward
• Stochastic recursive inclusion in two timescales with an application to the Lagrangian dual problem
• Two-timescale simultaneous perturbation stochastic approximation using deterministic perturbation sequences
• scientific article; zbMATH DE number 883236 (Why is no real title available?)
• Finite-time analysis and restarting scheme for linear two-time-scale stochastic approximation
• Convergence of stochastic approximation via martingale and converse Lyapunov methods
• A Two-Time-Scale Stochastic Optimization Framework with Applications in Control and Reinforcement Learning
• Convergence of stochastic approximation via martingale and converse Lyapunov methods
• Stochastic approximation with two time scales: the general case
• Regularization and two time scales for convergence of reinforcement learning
• Reinforcement learning in convergently non-stationary environments: feudal hierarchies and learned representations
• The ODE method for stochastic approximation and reinforcement learning with Markovian noise
• Sequential online subsampling for thinning experimental designs

This page was built for publication: A stability criterion for two timescale stochastic approximation schemes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2409333)