Error bounds for constant step-size Q-learning

DOI10.1016/J.SYSCONLE.2012.08.014zbMATH Open1255.93129OpenAlexW1999254175MaRDI QIDQ1932736FDOQ1932736

Authors Yanyan Li

Publication date 21 January 2013

Published in Systems \& Control Letters (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1016/j.sysconle.2012.08.014

zbMATH Keywords

stochastic approximation\(Q\)-learningMarkov decision processes

Mathematics Subject Classification ID

Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) (60J20) Learning and adaptive systems in artificial intelligence (68T05) Stochastic systems in control theory (general) (93E03)

Recommendations

• A generalization error for Q-learning
• Asymptotic analysis of temporal-difference learning algorithms with constant step-sizes
• Asymptotic analysis of temporal-difference learning algorithms with constant step-sizes
• \({\mathcal Q}\)-learning
• Boundedness of iterates in \(Q\)-learning

Cites work

• scientific article; zbMATH DE number 5957196 (Why is no real title available?)
• scientific article; zbMATH DE number 5348356 (Why is no real title available?)
• scientific article; zbMATH DE number 1043533 (Why is no real title available?)
• Asynchronous stochastic approximation and Q-learning
• Boundedness of iterates in \(Q\)-learning
• On the Convergence of Stochastic Iterative Dynamic Programming Algorithms
• Q-learning and enhanced policy iteration in discounted dynamic programming
• Q-learning and policy iteration algorithms for stochastic shortest path problems
• The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning
• \({\mathcal Q}\)-learning

Cited in

(13)

• Recent advances in reinforcement learning in finance
• Advances in Artificial Intelligence
• Asymptotic analysis of temporal-difference learning algorithms with constant step-sizes
• Some limit properties of Markov chains induced by recursive stochastic algorithms
• Data-driven approximate Q-learning stabilization with optimality error bound analysis
• A Discrete-Time Switching System Analysis of Q-Learning
• scientific article; zbMATH DE number 1804127 (Why is no real title available?)
• A generalization error for Q-learning
• Boundedness of iterates in \(Q\)-learning
• Convergence of Recursive Stochastic Algorithms Using Wasserstein Divergence
• Finite-sample analysis of nonlinear stochastic approximation with applications in reinforcement learning
• Q-learning for continuous-time linear systems: A model-free infinite horizon optimal control approach
• Settling the sample complexity of model-based offline reinforcement learning

Describes a project that uses

Uses Software

• Approxrl

This page was built for publication: Error bounds for constant step-size \(Q\)-learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1932736)