Discrete-Time Markov Chains
From MaRDI portal
Publication:4831226
DOI10.1007/b138226zbMath1072.60002OpenAlexW4299639519MaRDI QIDQ4831226
Publication date: 22 December 2004
Published in: Stochastic Modelling and Applied Probability (Search for Journal in Brave)
Full work available at URL: https://doi.org/10.1007/b138226
Production models (90B30) Dynamic programming (90C39) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10) Optimal stochastic control (93E20) Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) (60J20) Markov and semi-Markov decision processes (90C40) Research exposition (monographs, survey articles) pertaining to operations research and mathematical programming (90-02) Research exposition (monographs, survey articles) pertaining to probability theory (60-02)
Related Items (46)
Discrete-Time Semi-Markov Random Evolutions – Average and Diffusion Approximation of Difference Equations and Additive Functionals ⋮ Dissipative control of a three-species food chain stochastic system with a hidden Markov chain ⋮ Numerical methods for controlled regime-switching diffusions and regime-switching jump diffusions ⋮ On the notion of weak stability and related issues of hybrid diffusion systems ⋮ Fast computation of stationary joint probability distribution of sparse Markov chains ⋮ Safety verification for regime-switching jump diffusions via barrier certificates ⋮ Stochastic competitive Lotka-Volterra ecosystems under partial observation: feedback controls for permanence and extinction ⋮ Sequences of random matrices modulated by a discrete-time Markov chain* ⋮ A Fluid-Diffusion-Hybrid Limiting Approximation for Priority Systems with Fast and Slow Customers ⋮ Flows of rare events for regularly perturbed semi-Markov processes ⋮ System identification: regime switching, unmodeled dynamics, and binary sensors ⋮ Asymptotic properties of hybrid random processes modulated by Markov chains ⋮ Stochastic systems arising from Markov modulated empirical measures ⋮ Pathwise convergence rates for numerical solutions of Markovian switching stochastic differential equations ⋮ Analytic approximations of queues with lightly- and heavily-correlated autoregressive service times ⋮ Discrete-time Markov chains with two-time scales and a countable state space: limit results and queueing applications ⋮ Convergence of Markov chain approximation on generalized HJB equation and its applications ⋮ A novel HMM-based learning framework for improving dynamic wireless push system performance ⋮ Integration by parts and martingale representation for a Markov chain ⋮ Option pricing in a regime-switching model using the fast Fourier transform ⋮ Asymptotic optimality for consensus-type stochastic approximation algorithms using iterate averaging ⋮ Perturbed Markov chains with damping component ⋮ Discrete-time hybrid control in Borel spaces ⋮ Adaptive stepsize selection for tracking in a regime-switching environment ⋮ Stochastic optimization algorithms for barrier dividend strategies ⋮ Ergodic theory for controlled Markov chains with stationary inputs ⋮ Discrete-Time Semi-Markov Random Evolutions and their Applications ⋮ Singular perturbation for the discounted continuous control of piecewise deterministic Markov processes ⋮ Asymptotic properties of consensus-type algorithms for networked systems with regime-switching topologies ⋮ Asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes. I. ⋮ Balanced realizations of regime-switching linear systems ⋮ A stochastic approximation algorithm for option pricing model calibration with a switchable market ⋮ Distribution of PageRank Mass Among Principle Components of the Web ⋮ Numerical solutions for jump-diffusions with regime switching ⋮ Stability of Discrete-Time Regime-Switching Dynamic Systems with Delays ⋮ Asymptotic properties of Markov-modulated random sequences with fast and slow timescales ⋮ Weak convergence of Markov-modulated random sequences ⋮ How does a stochastic optimization/approximation algorithm adapt to a randomly evolving optimum/root with jump Markov sample paths ⋮ Coupling and ergodic theorems for Markov chains with damping component ⋮ Tracking and identification of regime-switching systems using binary sensors ⋮ Bounding the lumping error in Markov chain dynamics ⋮ Asymptotic expansions for power-exponential moments of hitting times for nonlinearly perturbed semi-Markov processes ⋮ Persistent tracking and identification of regime-switching systems with structural uncertainties: unmodeled dynamics, observation bias, and nonlinear model mismatch ⋮ From random Poincaré maps to stochastic mixed-mode-oscillation patterns ⋮ Optimal stopping of Markov switching Lévy processes ⋮ Sign-error adaptive filtering algorithms involving Markovian parameters
This page was built for publication: Discrete-Time Markov Chains
