Ergodic theory for controlled Markov chains with stationary inputs

DOI10.1214/17-AAP1300zbMATH Open1388.60130arXiv1604.04013OpenAlexW3099344026WikidataQ130161357 ScholiaQ130161357MaRDI QIDQ1751963FDOQ1751963

Authors: Yue Chen, Ana Bušić, Sean P. Meyn 

Publication date: 25 May 2018

Published in: The Annals of Applied Probability (Search for Journal in Brave)

Abstract: Consider a stochastic process $X(t)$ on a finite state space $sfX=1,dots,d$. It is conditionally Markov, given a real-valued `input process' $zeta(t)$. This is assumed to be small, which is modeled through the scaling, [ zeta_t = varepsilon zeta^1_t, qquad 0le varepsilon le 1,, ] where $zet{a}^1(t)$ is a bounded stationary process. The following conclusions are obtained, subject to smoothness assumptions on the controlled transition matrix and a mixing condition on $zeta(t)$: (i) A stationary version of the process is constructed, that is coupled with a stationary version of the Markov chain $$(t)}obtained with $zeta(t)equiv0$. The triple $$ is a jointly stationary process satisfying [ {sf P}{X(t) eq X^�ullet(t)} = O(varepsilon) ] Moreover, a second-order Taylor-series approximation is obtained: [ {sf P}{X(t) =i } ={sf P}{X^�ullet(t) =i } + varepsilon^2 varrho(i) + o(varepsilon^2),quad 1le ile d, ] with an explicit formula for the vector $varrhoinmathbb{R}^d$. (ii) For any $mge1$ and any function $fcolon1,dots,dimesmathbbRomathbb{R}^m$, the stationary stochastic process $Y(t)=f(X(t),zeta(t))$ has a power spectral density $ext{S}_f$ that admits a second order Taylor series expansion: A function $ext{S}_f^{(2)}colon[-pi,pi]omathbb{C}^{mimesm}$ is constructed such that [ ext{S}_f( heta) = ext{S}^�ullet_f( heta) + varepsilon^2 ext{S}_f^{(2)}( heta) + o(varepsilon^2),quad hetain [-pi,pi] . ] An explicit formula for the function $ext{S}_f^{(2)}$ is obtained, based in part on the bounds in (i). The results are illustrated using a version of the timing channel of Anantharam and Verdu.

Full work available at URL: https://arxiv.org/abs/1604.04013

Recommendations

• scientific article; zbMATH DE number 2015359
• scientific article; zbMATH DE number 4004053
• Ergodic control of a singularly perturbed Markov process in discrete time with general state and compact action spaces
• Stability of Markovian processes II: continuous-time processes and sampled chains
• Ergodic control of diffusion processes.

zbMATH Keywords

ergodic theoryinformation theorycontrolled Markov chain

Mathematics Subject Classification ID

Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) (60J20) Information theory (general) (94A15) Stationary stochastic processes (60G10) Performance evaluation, queueing, and scheduling in the context of computer systems (68M20)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Discrete-Time Markov Chains
• Large-Population Cost-Coupled LQG Problems With Nonuniform Agents: Individual-Mass Behavior and Decentralized $\varepsilon$-Nash Equilibria
• Perturbation theory and finite Markov chains
• Mean field games. I: The stationary case
• Title not available (Why is that?)
• AN OVERVIEW OF SOME STOCHASTIC STABILITY METHODS(<Special Issue>Network Design, Control and Optimization)
• Bits through queues
• Title not available (Why is that?)
• Ancillary Service to the Grid Using Intelligent Deferrable Loads
• State Estimation for the Individual and the Population in Mean Field Control With Application to Demand Dispatch
• Stability and approximation of invariant measures of Markov chains in random environments

This page was built for publication: Ergodic theory for controlled Markov chains with stationary inputs

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1751963)