Approximation of Markov Processes by Lower Dimensional Processes via Total Variation Metrics

DOI10.1109/TAC.2016.2578299MaRDI QIDQ5282358zbMATH OpenFDO

Authors I. Tzortzis, Themistoklis Charalambous, Christoforos N. Hadjicostis, Mikael Johansson, Charalambos D. Charalambous

Publication date 27 July 2017

Published in IEEE Transactions on Automatic Control (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1410.3976

Mathematics Subject Classification ID

Convergence of probability measures (60B10) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10)

Abstract: The aim of this paper is to approximate a finite-state Markov process by another process with fewer states, called herein the approximating process. The approximation problem is formulated using two different methods. The first method, utilizes the total variation distance to discriminate the transition probabilities of a high dimensional Markov process and a reduced order Markov process. The approximation is obtained by optimizing a linear functional defined in terms of transition probabilities of the reduced order Markov process over a total variation distance constraint. The transition probabilities of the approximated Markov process are given by a water-filling solution. The second method, utilizes total variation distance to discriminate the invariant probability of a Markov process and that of the approximating process. The approximation is obtained via two alternative formulations: (a) maximizing a functional of the occupancy distribution of the Markov process, and (b) maximizing the entropy of the approximating process invariant probability. For both formulations, once the reduced invariant probability is obtained, which does not correspond to a Markov process, a further approximation by a Markov process is proposed which minimizes the Kullback-Leibler divergence. These approximations are given by water-filling solutions. Finally, the theoretical results of both methods are applied to specific examples to illustrate the methodology, and the water-filling behavior of the approximations.

Recommendations

• Approximation of Markov semigroups in total variation distance
• Multivariate approximation in total variation. I: Equilibrium distributions of Markov jump processes
• Approximation of Markov semigroups in total variation distance under an irregular setting: an application to the CIR process
• Convergence in total variation distance for (in)homogeneous Markov processes
• scientific article; zbMATH DE number 4026480
• Approximation of measures by Markov processes and homogeneous affine iterated function systems
• Quantitative bounds for Markov chain convergence: Wasserstein and total variation distances
• Markov-Type Inequalities with Applications in Multivariate Approximation Theory
• An approximation theorem for Markov processes
• Approximation of symmetrizations by Markov processes

Cited in

(1)

• The average cost of Markov chains subject to total variation distance uncertainty

This page was built for publication: Approximation of Markov Processes by Lower Dimensional Processes via Total Variation Metrics

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5282358)