On reachability of Markov chains: A long-run average approach

DOI10.1109/TAC.2021.3071334MaRDI QIDQ6302386arXivFDO

Authors Daniel Ávila, Mauricio Junca

Publication date 31 May 2018

Published in arXiv

Full work available at URL https://doi.org/10.1109/TAC.2021.3071334

Mathematics Subject Classification ID

Markov and semi-Markov decision processes (90C40) Attainable sets, reachability (93B03) Controllability (93B05) Stochastic systems in control theory (general) (93E03)

arXiv classification

math.OC

Abstract: We consider a Markov control model in discrete time with countable both state space and action space. Using the value function of a suitable long-run average reward problem, we study various reachability/controllability problems. First, we characterize the domain of attraction and escape set of the system, and a generalization called $p$-domain of attraction, using the aforementioned value function. Next, we solve the problem of maximizing the probability of reaching a set $A$ while avoiding a set $B$. Finally, we consider a constrained version of the previous problem where we ask for the probability of reaching the set $B$ to be bounded. In the finite case, we use linear programming formulations to solve these problems. Finally, we apply our results to a example of an object that navigates under stochastic influence.

This page was built for publication: On reachability of Markov chains: A long-run average approach

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6302386)