Linear stochastic approximation driven by slowly varying Markov chains

DOI10.1016/S0167-6911(03)00132-4MaRDI QIDQ2503529zbMATH OpenOpenAlexFDO

Authors Vijay R. Konda, John N. Tsitsiklis

Publication date 21 September 2006

Published in Systems \& Control Letters (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1016/s0167-6911(03)00132-4

zbMATH Keywords

Stochastic approximationAdaptive algorithmsReinforcement learning

Mathematics Subject Classification ID

Stochastic approximation (62L20) Identification in stochastic control theory (93E12)

Recommendations

• Convergence of Markovian stochastic approximation with discontinuous dynamics
• Stochastic approximation with `controlled Markov' noise
• Finite-sample analysis of nonlinear stochastic approximation with applications in reinforcement learning
• Adaptive stochastic approximation algorithm
• Two Time-Scale Stochastic Approximation with Controlled Markov Noise and Off-Policy Temporal-Difference Learning

Cites work

• scientific article; zbMATH DE number 48727 (Why is no real title available?)
• scientific article; zbMATH DE number 1321699 (Why is no real title available?)
• scientific article; zbMATH DE number 976356 (Why is no real title available?)
• scientific article; zbMATH DE number 1043533 (Why is no real title available?)
• Asymptotic behavior of stochastic approximation and large deviations
• Convergence of an adaptive linear estimation algorithm
• Markov chains and stochastic stability
• OnActor-Critic Algorithms
• Stochastic approximation with two time scales
• The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning

Cited in

(6)

• Finite-sample analysis of nonlinear stochastic approximation with applications in reinforcement learning
• Approximation order analysis for the piecewise linear Markov method
• Simulation-based optimal sensor scheduling with application to observer trajectory planning
• Two Time-Scale Stochastic Approximation with Controlled Markov Noise and Off-Policy Temporal-Difference Learning
• Two-timescale stochastic gradient descent in continuous time with applications to joint online parameter estimation and optimal sensor placement
• Finite-time analysis and restarting scheme for linear two-time-scale stochastic approximation

This page was built for publication: Linear stochastic approximation driven by slowly varying Markov chains

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2503529)