An emphatic approach to the problem of off-policy temporal-difference learning

MaRDI QIDQ2810885zbMATH OpenFDO

Authors Richard S. Sutton, A. Rupam Mahmood, Martha White

Publication date 6 June 2016

Published in Journal of Machine Learning Research (JMLR) (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1503.04269, http://jmlr.csail.mit.edu/papers/v17/14-488.html

zbMATH Keywords

convergencestabilityfunction approximationoff-policy learningtemporal-difference learning

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Recommendations

• Weak convergence properties of constrained emphatic temporal-difference learning with constant and slowly diminishing stepsize
• \(\text{Q}(\lambda)\) with off-policy corrections
• A finite time analysis of temporal difference learning with linear function approximation

Cited in

(11)

• Policy evaluation with temporal differences: a survey and comparison
• Estimating Optimal Infinite Horizon Dynamic Treatment Regimes via pT-Learning
• Online Bootstrap Inference For Policy Evaluation In Reinforcement Learning
• Adaptive importance sampling for value function approximation in off-policy reinforcement learning
• Gradient temporal-difference learning for off-policy evaluation using emphatic weightings
• Efficiently Breaking the Curse of Horizon in Off-Policy Evaluation with Double Reinforcement Learning
• Multi-agent reinforcement learning: a selective overview of theories and algorithms
• Statistical inference for online decision making via stochastic gradient descent
• \(\text{Q}(\lambda)\) with off-policy corrections
• Distributed consensus-based multi-agent temporal-difference learning
• Off-policy temporal difference learning with distribution adaptation in fast mixing chains

This page was built for publication: An emphatic approach to the problem of off-policy temporal-difference learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2810885)