Equilibrium in misspecified Markov decision processes

DOI10.3982/TE3843zbMATH Open1475.91181arXiv1502.06901OpenAlexW3157638780MaRDI QIDQ5164471FDOQ5164471

Authors: Ignacio Esponda, Demian Pouzo 

Publication date: 11 November 2021

Published in: Theoretical Economics (Search for Journal in Brave)

Abstract: We study Markov decision problems where the agent does not know the transition probability function mapping current states and actions to future states. The agent has a prior belief over a set of possible transition functions and updates beliefs using Bayes' rule. We allow her to be misspecified in the sense that the true transition probability function is not in the support of her prior. This problem is relevant in many economic settings but is usually not amenable to analysis by the researcher. We make the problem tractable by studying asymptotic behavior. We propose an equilibrium notion and provide conditions under which it characterizes steady state behavior. In the special case where the problem is static, equilibrium coincides with the single-agent version of Berk-Nash equilibrium (Esponda and Pouzo (2016)). We also discuss subtle issues that arise exclusively in dynamic settings due to the possibility of a negative value of experimentation.

Full work available at URL: https://arxiv.org/abs/1502.06901

Recommendations

• Markov equilibria in discounted stochastic games
• Exact estimation for Markov chain equilibrium expectations
• Stationary Markov Equilibria
• Conditional Markov equilibria in discounted dynamic games
• Time-inconsistent risk-sensitive equilibrium for countable-stated Markov decision processes
• Stationary Markov perfect equilibria in discounted stochastic games
• Markov perfect equilibria in a dynamic decision model with quasi-hyperbolic discounting
• Stochastic equilibria
• Stationary almost Markov perfect equilibria in discounted stochastic games

zbMATH Keywords

equilibriumMarkov decision processmisspecified model

Mathematics Subject Classification ID

Markov and semi-Markov decision processes (90C40) Special types of economic equilibria (91B52)

Cited In (6)

• Pathwise concentration bounds for Bayesian beliefs
• Robust persuasion of a privately informed receiver
• Asymptotic behavior of Bayesian learners with misspecified models
• Berk-Nash Equilibrium: A Framework for Modeling Agents With Misspecified Models
• On existence of Berk-Nash equilibria in misspecified Markov decision processes with infinite spaces
• A reputation for honesty

This page was built for publication: Equilibrium in misspecified Markov decision processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5164471)