Bayesian learning of noisy Markov decision processes

DOI10.1145/2414416.2414420zbMATH Open1384.62104arXiv1211.5901OpenAlexW2001339644MaRDI QIDQ4635207FDOQ4635207

Sumeetpal S. Singh, Nicolas Chopin, Nick Whiteley

Publication date: 16 April 2018

Published in: ACM Transactions on Modeling and Computer Simulation (Search for Journal in Brave)

Abstract: We consider the inverse reinforcement learning problem, that is, the problem of learning from, and then predicting or mimicking a controller based on state/action data. We propose a statistical model for such data, derived from the structure of a Markov decision process. Adopting a Bayesian approach to inference, we show how latent variables of the model can be estimated, and how predictions about actions can be made, in a unified framework. A new Markov chain Monte Carlo (MCMC) sampler is devised for simulation from the posterior distribution. This step includes a parameter expansion step, which is shown to be essential for good convergence properties of the MCMC sampler. As an illustration, the method is applied to learning a human controller.

Full work available at URL: https://arxiv.org/abs/1211.5901

Recommendations

• Inverse reinforcement learning from summary data
• A Bayesian approach for learning and planning in partially observable Markov decision processes
• Inverse reinforcement learning in partially observable environments
• Learning parametric policies and transition probability models of Markov decision processes from data
• Markov decision processes with arbitrary reward processes

zbMATH Keywords

Bayesian inferencedata augmentationMarkov decision processparameter expansionMarkov chain Monte Carlo sampler

Mathematics Subject Classification ID

Bayesian inference (62F15) Monte Carlo methods (65C05) Learning and adaptive systems in artificial intelligence (68T05)

Cited In (5)

• Title not available (Why is that?)
• Bayesian Representation of Stochastic Processes under Learning: de Finetti Revisited
• Playing to train your video game avatar
• Learning Variable-Length Markov Models of Behavior
• Learning parametric policies and transition probability models of Markov decision processes from data

This page was built for publication: Bayesian learning of noisy Markov decision processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4635207)