Markov chains and stochastic stability

Publication date: 2 November 1993

Published in: Communications and Control Engineering (Search for Journal in Brave)

Introductory exposition (textbooks, tutorial papers, etc.) pertaining to probability theory (60-01) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10) Stochastic stability in control theory (93E15) Continuous-time Markov processes on discrete state spaces (60J27)

Related Items (only showing first 100 items - show all)

Moment and probability inequalities for sums of bounded additive functionals of regular Markov chains via the Nummelin splitting technique ⋮ Hoeffding's inequality for uniformly ergodic Markov chains ⋮ A note on a simple Markov bilinear stochastic process ⋮ Geometric ergodicity of Metropolis algorithms ⋮ Moment conditions for a sequence with negative drift to be uniformly bounded in \(L^r\) ⋮ Stability of nonlinear AR(1) time series with delay ⋮ The sample ACF of a simple bilinear process ⋮ Some dichotomy results for functionals of Harris recurrent Markov chains ⋮ A quasi-ergodic theorem for evanescent processes ⋮ A criterion on asymptotic stability for partially equicontinuous Markov operators ⋮ Geometric ergodicity of Pólya-Gamma Gibbs sampler for Bayesian logistic regression with a flat prior ⋮ Adaptive MCMC for multiple changepoint analysis with applications to large datasets ⋮ Noise contrastive estimation: asymptotic properties, formal comparison with MC-MLE ⋮ Bayesian Dirichlet mixture model for multivariate extremes: a re-parametrization ⋮ Efficient estimation of the link function parameter in a robust Bayesian binary regression model ⋮ Parameter estimation via stochastic variants of the ECM algorithm with applications to plant growth modeling ⋮ A modified conditional Metropolis-Hastings sampler ⋮ A Monte Carlo approach to quantifying model error in Bayesian parameter estimation ⋮ On the martingale problem and Feller and strong Feller properties for weakly coupled Lévy type operators ⋮ Weighted exponential random graph models: scope and large network limits ⋮ Rates of convergence for everywhere-positive Markov chains ⋮ Sliding mode control for state-delayed Markov jump systems with partly unknown transition probabilities ⋮ Outperforming the Gibbs sampler empirical estimator for nearest-neighbor random fields ⋮ A gambling system and a Markov chain ⋮ The random difference equation \(X_ n = A_ n X_{n-1} + B_ n\) in the critical case ⋮ Dam processes with state dependent batch sizes and intermittent production processes with state dependent rates ⋮ Ergodic and light traffic properties of a complex repairable system ⋮ On the role of Rouché's theorem in queueing analysis ⋮ Approximation of average cost optimal policies for general Markov decision processes with unbounded costs ⋮ A strategic market game with secured lending ⋮ About the multidimensional competitive learning vector quantization algorithm with constant gain ⋮ Invariant probabilities for Markov chains on a metric space ⋮ Coupling with compensators ⋮ Tails of passage-times and an application to stochastic processes with boundary reflection in wedges ⋮ Numerical non-identifiability regions of the minimal model of glucose kinetics: Superiority of Bayesian estimation ⋮ Aspiration adaptation in the ultimatum minigame. ⋮ On optimal inventory control with independent stochastic item returns. ⋮ SPRT and CUSUM in hidden Markov models ⋮ Saddlepoint approximations and nonlinear boundary crossing probabilities of Markov random walks ⋮ Nonexistence of a class of variate generation schemes. ⋮ Irreducibility results and conservativity of the absolute difference chain ⋮ On single versus multiple imputation for a class of stochastic algorithms estimating maximum likelihood ⋮ Control of end-to-end delay tails in a multiclass network: LWDF discipline optimality ⋮ Honest exploration of intractable probability distributions via Markov chain Monte Carlo. ⋮ Stability and instability of a two-station queueing network ⋮ Phase transitions and metastability in Markovian and molecular systems ⋮ Speed of stochastic locally contractive systems. ⋮ Convergence of the Monte Carlo expectation maximization for curved exponential families. ⋮ Forgetting the initial distribution for hidden Markov models ⋮ On universal transfer learning ⋮ Hastings-Metropolis algorithms and reference measures ⋮ Integral equation solutions as prior distributions for Bayesian model selection ⋮ Evolving aspirations and cooperation ⋮ Characterizations of overtaking optimality for controlled diffusion processes ⋮ Learning from uniformly ergodic Markov chains ⋮ Approximate regenerative-block bootstrap for Markov chains ⋮ Micro-local analysis for the Metropolis algorithm ⋮ Estimation for a class of positive nonlinear time series models ⋮ Geometric convergence of the Metropolis-Hastings simulation algorithm ⋮ Geometric ergodicity of Gibbs and block Gibbs samplers for a hierarchical random effects model ⋮ On polynomial mixing bounds for stochastic differential equations ⋮ Particle filtering with path sampling and an application to a bimodal ocean current model ⋮ Zero-sum ergodic semi-Markov games with weakly continuous transition probabilities ⋮ How often does a Harris recurrent Markov chain recur? ⋮ Exact asymptotics for the stationary distribution of a Markov chain: a production model ⋮ Experimentation, imitation, and stochastic stability ⋮ Nearest neighbor conditional estimation for Harris recurrent Markov chains ⋮ Polarity formation by a higher order interaction Markov-like chain ⋮ Balanced partitions for Markov chains ⋮ On geometric ergodicity of the MTAR process ⋮ Instability of FIFO in a simple queueing system with arbitrarily low loads ⋮ On the stationary distribution of queue lengths in a multi-class priority queueing system with customer transfers ⋮ A strong approximation theorem for stochastic recursive algorithms ⋮ Single sample path-based optimization of Markov chains ⋮ Uniform time average consistency of Monte Carlo particle filters ⋮ Asymptotically homogeneous iterated random functions with applications to the HARCH process ⋮ Importance sampling for families of distributions ⋮ Genealogical processes for Fleming-Viot models with selection and recombination ⋮ State-dependent Foster-Lyapunov criteria for subgeometric convergence of Markov chains ⋮ Asymptotics of first passage times for random walk in an orthant ⋮ On convergence rates of Gibbs samplers for uniform distributions ⋮ Two convergence properties of hybrid samplers ⋮ How hot can a heat bath get? ⋮ Ergodicity of dissipative differential equations subject to random impulses ⋮ \(V\)-subgeometric ergodicity for a Hastings-Metropolis algorithm ⋮ Invariant probability measures for a class of Feller Markov chains ⋮ A note on the stationarity of a threshold first-order bilinear process ⋮ Verifying irreducibility and continuity of a nonlinear time series ⋮ Asymptotics of a class of \(p\)th-order nonlinear autoregressive processes ⋮ Random motions, classes of ergodic Markov chains and beta distributions ⋮ A direct approach to conformational dynamics based on hybrid Monte Carlo ⋮ Information bounds for Gibbs samplers ⋮ Lyapunov functions for random walks and strings in random environment ⋮ Large deviation lower bounds for arbitrary additive functionals of a Markov chain ⋮ Time and Palm stationarity of repairable systems ⋮ On the use of Lyapunov methods in renewal theory ⋮ Bounds on regeneration times and convergence rates for Markov chains ⋮ Jump-diffusion Markov processes on orthogonal groups for object pose estimation ⋮ Topological conditions enabling use of Harris methods in discrete and continuous time ⋮ Perfect sampling from independent Metropolis-Hastings chains

This page was built for publication: Markov chains and stochastic stability