Reinforcement Learning, Bit by Bit

DOI10.1561/2200000097zbMATH Open1525.68120arXiv2103.04047OpenAlexW4383982036MaRDI QIDQ6139546FDOQ6139546

Authors: Benjamin Van Roy, Vikranth R. Dwaracherla, Morteza Ibrahimi, Ian Osband, Zheng Wen 

Publication date: 19 December 2023

Published in: Foundations and Trends® in Machine Learning (Search for Journal in Brave)

Abstract: Reinforcement learning agents have demonstrated remarkable achievements in simulated environments. Data efficiency poses an impediment to carrying this success over to real environments. The design of data-efficient agents calls for a deeper understanding of information acquisition and representation. We discuss concepts and regret analysis that together offer principled guidance. This line of thinking sheds light on questions of what information to seek, how to seek that information, and what information to retain. To illustrate concepts, we design simple agents that build on them and present computational results that highlight data efficiency.

Full work available at URL: https://arxiv.org/abs/2103.04047

Recommendations

• Reinforcement learning in sparse-reward environments with hindsight policy gradients
• Reinforcement learning agents
• scientific article; zbMATH DE number 1977952
• Reinforcement learning
• The effect of representation and knowledge on goal-directed exploration with reinforcement-learning algorithms

zbMATH Keywords

reinforcement learning

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Research exposition (monographs, survey articles) pertaining to computer science (68-02)

Cites Work

• 10.1162/153244303765208377
• Elements of Information Theory
• Asymptotically efficient adaptive allocation rules
• Title not available (Why is that?)
• Finite-time analysis of the multiarmed bandit problem
• A Definition of Subjective Probability
• Regret analysis of stochastic and nonstochastic multi-armed bandit problems
• Reinforcement learning. An introduction
• Adaptive treatment allocation and the multi-armed bandit problem
• The knowledge gradient algorithm for a general class of online learning problems
• Title not available (Why is that?)
• An Adaptive Sampling Algorithm for Solving Markov Decision Processes
• Near-optimal reinforcement learning in polynomial time
• Practical issues in temporal difference learning
• Learning to optimize via posterior sampling
• Some upper bounds for relative entropy and applications
• An adaptive optimal controller for discrete-time Markov environments
• Bayesian reinforcement learning: a survey
• An information-theoretic analysis of Thompson sampling
• A Tutorial on Thompson Sampling
• Learning to optimize via information-directed sampling
• Deep exploration via randomized value functions
• Satisficing in Time-Sensitive Bandit Learning

Cited In (3)

• Finding the optimal exploration-exploitation trade-off online through Bayesian risk estimation and minimization
• Occupancy information ratio: infinite-horizon, information-directed, parameterized policy search
• Reinforcement learning in non-Markovian environments

This page was built for publication: Reinforcement Learning, Bit by Bit

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6139546)