Stationary Distributions of Discrete-Time Markov Chains in Random Environment: Exact Computations and Bounds
Let \(\pi=(\pi(e,x) : e\in E, x\in X)\) be the joint stationary distribution of a Markov chain on \(E\times X\) with transition probabilities given by \[ p((e,x),(e',x')) = \begin{cases} q(e)p_{X| E}(x,x'\mid e) + r(e) p_E(e,e) \alpha(x'\mid e,x,e) &\text{ if }e'=e,\\ r(e) p_E(e,e')\alpha(x'\mid e,x,e') &\text{ if }e'\neq e,\end{cases} \] \(e\) is the current environmental state and \(r(e) = 1- q(e)\) is the probability that there is an environmental change. The first result is an inversion formula which gives \(\pi(e,x)\) in terms of a \(q(e)\)-potential and \(\pi_{a(e)}(x)\), the probability that chain is at \(x\) just after an environmental change to \(e\). This serves as the basis for an efficient computational procedure for obtaining explicit expressions of the stationary distributions of many models. Necessary and sufficient conditions are given for the stationary distribution to be of the form \(\pi(e,x)=\pi_E(e)\pi_{X\mid E}(x\mid e)\). Product-form stationary distribution can be used to approximate the stationary distribution of intricate Markov chains in slowly varying random environment. The paper ends with expressions for error bounds and three illustrations of the applicability of the method introduced. The first one is the early arrival queue with periodically varying arrival/service probabilities.
- Generalized product-form stationary distributions for Markov chains in random environments with queueing applications
- Calculating the characteristics of a queue in a synchronous random Markov environment
- Separable solutions for Markov processes in random environments
- Stationary distributions and convergence for \(M/M/1\) queues in interactive random environment
- Markovian queueing networks in a random environment
- A bivariate random environmental stress model
- A heterogeneous blocking system in a random environment
- Aggregation of Markov chains
- An Analytic Approach to a General Class of G/G/s Queueing Systems
- An analytical solution for a tandem queue with blocking
- Connecting internally balanced quasi-reversible Markov processes
- Connecting reversible Markov processes
- Decomposability in queues with background states
- Finite birth-and-death models in randomly changing environments
- scientific article; zbMATH DE number 3736679 (Why is no real title available?)
- scientific article; zbMATH DE number 3736680 (Why is no real title available?)
- scientific article; zbMATH DE number 1324228 (Why is no real title available?)
- Interconnections of Markov chains and quasi-reversible queuing networks
- Introduction to stochastic networks
- Markov chains. Gibbs fields, Monte Carlo simulation, and queues
- On determining absorption probabilities for Markov chains in random environments
- On the association of the lifelengths of components subjected to a stochastic environment
- On the limiting distribution of a supercritical branching process in a random environment
- On the transient delays of M/G/1 queues
- Optimal admission control of markov-modulated batch arrivals to a finite-capacity buffer
- Optimal control of arrivals to multiserver queues in a random environment
- Perturbation analysis of the M/M/1 queue in a markovian environment via the matrix-geometric method
- Spectral expansion solution for a class of Markov models: application and comparison with the matrix-geometric method
- The M/M/∞ queue in a random environment
- Calculating the characteristics of a queue in a synchronous random Markov environment
- Constructions of Markov processes in random environments which lead to a product form of the stationary measure
- Generalized product-form stationary distributions for Markov chains in random environments with queueing applications
- Invariance of stationary ergodic distributions of homogeneous Markov processes under random interrupts
This page was built for publication: Stationary Distributions of Discrete-Time Markov Chains in Random Environment: Exact Computations and Bounds
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4451558)
