Reversible Markov decision processes and the Gaussian free field

DOI10.1016/J.SYSCONLE.2022.105382MaRDI QIDQ2107621zbMATH OpenOpenAlexWikidataFDO

Authors Venkat Anantharam

Publication date 2 December 2022

Published in Systems \& Control Letters (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2207.05217

zbMATH Keywords

biconnectednessGaussian free fieldMarkov chain Monte CarloMarkov decision processreversibility

Mathematics Subject Classification ID

Markov and semi-Markov decision processes (90C40)

Abstract: A Markov decision problem is called reversible if the stationary controlled Markov chain is reversible under every stationary Markovian strategy. A natural application in which such problems arise is in the control of Metropolis-Hastings type dynamics. We characterize all discrete time reversible Markov decision processes with finite state and actions spaces. We show that policy iteration algorithm for finding an optimal policy can be significantly simplified Markov decision problems of this type. We also highlight the relation between the finite time evolution of the accrual of reward and the Gaussian free field associated to the controlled Markov chain.

Recommendations

• Reversible Markov Decision Processes with an Average-Reward Criterion
• scientific article; zbMATH DE number 3873074
• Markov decision processes
• Markovian decision programming with recursive vector-reward
• Generalized Inverses in Discrete Time Markov Decision Processes

Cites work

• Discrete-Time Controlled Markov Processes with Average Cost Criterion: A Survey
• Gaussian Hilbert Spaces
• Graph theory
• Markov paths, loops and fields. École d'Été de Probabilités de Saint-Flour XXXVIII -- 2008
• Probability on trees and networks
• Reversible Markov Decision Processes with an Average-Reward Criterion

Cited in

(2)

• Reversibility and equivalence in directed Markov fields
• Reversible Markov Decision Processes with an Average-Reward Criterion

This page was built for publication: Reversible Markov decision processes and the Gaussian free field

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2107621)