Bayesian reinforcement learning: a survey
From MaRDI portal
Abstract: Bayesian methods for machine learning have been widely investigated, yielding principled methods for incorporating prior information into inference algorithms. In this survey, we provide an in-depth review of the role of Bayesian methods for the reinforcement learning (RL) paradigm. The major incentives for incorporating Bayesian reasoning in RL are: 1) it provides an elegant approach to action-selection (exploration/exploitation) as a function of the uncertainty in learning; and 2) it provides a machinery to incorporate prior knowledge into the algorithms. We first discuss models and methods for Bayesian inference in the simple single-step Bandit model. We then review the extensive recent literature on Bayesian methods for model-based RL, where prior information can be expressed on the parameters of the Markov model. We also present Bayesian methods for model-free RL, where priors are expressed over the value function or policy class. The objective of the paper is to provide a comprehensive survey on Bayesian RL algorithms and their theoretical and empirical properties.
Both Bayesian and reinforcement learning methods are well known and in common use in machine learning. The text presents an updated survey on how to use the Bayesian paradigm in reinforcement learning. Both model-based and model-free Bayesian reinforcement learning paradigms are presented, along with risk-aware Bayesian reinforcement learning. The text is addressed to students and researchers interested in Bayesian reinforcement learning.
Recommendations
- Dual control for approximate Bayesian reinforcement learning
- A Bayesian approach for learning and planning in partially observable Markov decision processes
- Bayesian Reinforcement Learning with Exploration
- Transfer learning for reinforcement learning domains: a survey
- Reinforcement learning theory, algorithms and its application
Cited in
(31)- Reinforcement learning, sequential Monte Carlo and the EM algorithm
- A model for system uncertainty in reinforcement learning
- Learning and planning in partially observable environments without prior domain knowledge
- Trading utility and uncertainty: applying the value of information to resolve the exploration-exploitation dilemma in reinforcement learning
- A new algorithm for the LQR problem with partially unknown dynamics
- A Bayesian perspective of statistical machine learning for big data
- A cooperative game for automated learning of elasto-plasticity knowledge graphs and models with AI-guided experimentation
- Personalized optimization with user's feedback
- Bayesian policy gradient and actor-critic algorithms
- Bayesian Reinforcement Learning with Exploration
- Inverse reinforcement learning via nonparametric spatio-temporal subgoal modeling
- Active Inference: Demystified and Compared
- Sophisticated inference
- scientific article; zbMATH DE number 7626804 (Why is no real title available?)
- scientific article; zbMATH DE number 7415125 (Why is no real title available?)
- On Overfitting and Asymptotic Bias in Batch Reinforcement Learning with Partial Observability
- Constrained multiagent Markov decision processes: a taxonomy of problems and algorithms
- Model-based Reinforcement Learning: A Survey
- A Bayesian reinforcement learning approach in Markov games for computing near-optimal policies
- Embedding active learning in batch-to-batch optimization using reinforcement learning
- Learning reward machines: a study in partially observable reinforcement learning
- Robust moving target defense against unknown attacks: a meta-reinforcement learning approach
- Optimal Treatment Regimes: A Review and Empirical Comparison
- Reward Maximization Through Discrete Active Inference
- Reinforcement Learning, Bit by Bit
- Modern Bayesian experimental design
- Finding the optimal exploration-exploitation trade-off online through Bayesian risk estimation and minimization
- Reinforcement learning
- Planning in artificial intelligence
- Statistical field theory for Markov decision processes under uncertainty
- Nonparametric additive value functions: interpretable reinforcement learning with an application to surgical recovery
This page was built for publication: Bayesian reinforcement learning: a survey
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2809805)
