Ergodic theory for controlled Markov chains with stationary inputs

DOI10.1214/17-AAP1300MaRDI QIDQ1751963zbMATH OpenOpenAlexWikidataFDO

Authors Yue Chen, Ana Bušić, Sean P. Meyn

Publication date 25 May 2018

Published in The Annals of Applied Probability (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1604.04013

zbMATH Keywords

controlled Markov chainergodic theoryinformation theory

Mathematics Subject Classification ID

Stationary stochastic processes (60G10) Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) (60J20) Performance evaluation, queueing, and scheduling in the context of computer systems (68M20) Information theory (general) (94A15)

Abstract: Consider a stochastic process $X(t)$ on a finite state space $sfX=1,dots,d$. It is conditionally Markov, given a real-valued `input process' $zeta(t)$. This is assumed to be small, which is modeled through the scaling, [ zeta_t = varepsilon zeta^1_t, qquad 0le varepsilon le 1,, ] where $zet{a}^1(t)$ is a bounded stationary process. The following conclusions are obtained, subject to smoothness assumptions on the controlled transition matrix and a mixing condition on $zeta(t)$: (i) A stationary version of the process is constructed, that is coupled with a stationary version of the Markov chain $$(t)}obtained with $zeta(t)equiv0$. The triple $$ is a jointly stationary process satisfying [ {sf P}{X(t) eq X^�ullet(t)} = O(varepsilon) ] Moreover, a second-order Taylor-series approximation is obtained: [ {sf P}{X(t) =i } ={sf P}{X^�ullet(t) =i } + varepsilon^2 varrho(i) + o(varepsilon^2),quad 1le ile d, ] with an explicit formula for the vector $varrhoinmathbb{R}^d$. (ii) For any $mge1$ and any function $fcolon1,dots,dimesmathbbRomathbb{R}^m$, the stationary stochastic process $Y(t)=f(X(t),zeta(t))$ has a power spectral density $ext{S}_f$ that admits a second order Taylor series expansion: A function $ext{S}_f^{(2)}colon[-pi,pi]omathbb{C}^{mimesm}$ is constructed such that [ ext{S}_f( heta) = ext{S}^�ullet_f( heta) + varepsilon^2 ext{S}_f^{(2)}( heta) + o(varepsilon^2),quad hetain [-pi,pi] . ] An explicit formula for the function $ext{S}_f^{(2)}$ is obtained, based in part on the bounds in (i). The results are illustrated using a version of the timing channel of Anantharam and Verdu.

Recommendations

• scientific article; zbMATH DE number 2015359
• scientific article; zbMATH DE number 4004053
• Ergodic control of a singularly perturbed Markov process in discrete time with general state and compact action spaces
• Stability of Markovian processes II: continuous-time processes and sampled chains
• Ergodic control of diffusion processes.

Cites work

• AN OVERVIEW OF SOME STOCHASTIC STABILITY METHODS(<Special Issue>Network Design, Control and Optimization)
• Ancillary Service to the Grid Using Intelligent Deferrable Loads
• Bits through queues
• Discrete-Time Markov Chains
• scientific article; zbMATH DE number 3909450 (Why is no real title available?)
• scientific article; zbMATH DE number 107482 (Why is no real title available?)
• scientific article; zbMATH DE number 193291 (Why is no real title available?)
• scientific article; zbMATH DE number 1158743 (Why is no real title available?)
• Large-Population Cost-Coupled LQG Problems With Nonuniform Agents: Individual-Mass Behavior and Decentralized \varepsilon-Nash Equilibria
• Mean field games. I: The stationary case
• Perturbation theory and finite Markov chains
• Stability and approximation of invariant measures of Markov chains in random environments
• State Estimation for the Individual and the Population in Mean Field Control With Application to Demand Dispatch

This page was built for publication: Ergodic theory for controlled Markov chains with stationary inputs

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1751963)