Multiple time scale decomposition of discrete time Markov chains

DOI10.1016/0167-6911(88)90075-8MaRDI QIDQ1111248zbMATH OpenOpenAlexFDO

Authors J. R. Rohlicek, Alan S. Willsky

Publication date 1988

Published in Systems \& Control Letters (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1016/0167-6911(88)90075-8

zbMATH Keywords

aggregationmultiple time scale decompositionperturbation theory

Mathematics Subject Classification ID

Continuous-time Markov processes on discrete state spaces (60J27)

Review text

The multiple time scale decomposition of discrete time, finite state Markov chains is addressed. In another paper by the same authors [J. Assoc. Comput. Mach. 35, No.3, 675-696 (1988; Zbl 0643.60057)], the behavior of a continuous time Markov chain is approximated using a fast time scale, \(\epsilon\)-independent, continuous time process, and a reduced order perturbed process. The procedure can then be iterated to obtain a complete multiple time scale decomposition. In the discrete time case presented in this paper, the basic approximation has a `hybrid' form. In this form, the fast time scale behavior is approximated using an \(\epsilon\)-independent discrete time Markov chain, and the slow behavior is captured by a perturbed, continuous time process.

Recommendations

• scientific article; zbMATH DE number 3881614
• A martingale decomposition of discrete Markov chains
• Large deviations of Markov chains with multiple time-scales
• Discrete-Time Markov Chains
• scientific article; zbMATH DE number 3858113
• On discrete-time stable multiple Markov processes
• A decomposition for Markov processes at an independent exponential time
• Time distributions in discrete time Markov processes
• Ergodic decomposition of Markov chains

Cites work

• Aggregation of Variables in Dynamic Systems
• Hierarchical aggregation of singularly perturbed finite state markov processes
• scientific article; zbMATH DE number 3574935 (Why is no real title available?)
• scientific article; zbMATH DE number 3239038 (Why is no real title available?)
• Multiple time scale decomposition of discrete time Markov chains
• Singularly perturbed difference equations in optimal control problems
• The reduction of perturbed Markov generators

Cited in

(5)

• Multiple time scale decomposition of discrete time Markov chains
• Singulary perturbed Markov control problem: Limiting average cost
• Estimating equilibrium probabilities for band diagonal Markov chains using aggregation and disaggregation techniques
• Asymptotic expansions for stationary distributions of perturbed semi-Markov processes
• scientific article; zbMATH DE number 3952015 (Why is no real title available?)

This page was built for publication: Multiple time scale decomposition of discrete time Markov chains

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1111248)