Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) (60J20) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10) Dynamic programming (90C39) Research exposition (monographs, survey articles) pertaining to probability theory (60-02) Research exposition (monographs, survey articles) pertaining to operations research and mathematical programming (90-02) Production models (90B30) Markov and semi-Markov decision processes (90C40) Optimal stochastic control (93E20)
Recommendations
- Continuous-time Markov chains and applications. A two-time-scale approach
- Markov Processes and Applications
- Markov chains and stochastic stability
- Continuous-time Markov decision processes. Borel space models and general control strategies. With a foreword by Albert Nikolaevich Shiryaev
- Markov chains. Theory and applications
Cited in
(50)- Convergence of Markov chain approximation on generalized HJB equation and its applications
- Stability of Discrete-Time Regime-Switching Dynamic Systems with Delays
- Singular perturbation for the discounted continuous control of piecewise deterministic Markov processes
- Distribution of PageRank Mass Among Principle Components of the Web
- Asymptotic optimality for consensus-type stochastic approximation algorithms using iterate averaging
- Asymptotic properties of Markov-modulated random sequences with fast and slow timescales
- Continuous-time Markov chains and applications. A two-time-scale approach
- Asymptotic properties of hybrid random processes modulated by Markov chains
- System identification: regime switching, unmodeled dynamics, and binary sensors
- Bounding the lumping error in Markov chain dynamics
- Flows of rare events for regularly perturbed semi-Markov processes
- Multiple time scale decomposition of discrete time Markov chains
- Pathwise convergence rates for numerical solutions of Markovian switching stochastic differential equations
- Dissipative control of a three-species food chain stochastic system with a hidden Markov chain
- Numerical solutions for jump-diffusions with regime switching
- Discrete-time hybrid control in Borel spaces
- Optimal stopping of Markov switching Lévy processes
- Perturbed Markov chains with damping component
- Ergodic theory for controlled Markov chains with stationary inputs
- Tracking and identification of regime-switching systems using binary sensors
- Discrete-time semi-Markov random evolutions -- average and diffusion approximation of difference equations and additive functionals
- Stochastic competitive Lotka-Volterra ecosystems under partial observation: feedback controls for permanence and extinction
- Balanced realizations of regime-switching linear systems
- Safety verification for regime-switching jump diffusions via barrier certificates
- Weak convergence of Markov-modulated random sequences
- Discrete-time semi-Markov random evolutions and their applications
- Option pricing in a regime-switching model using the fast Fourier transform
- Asymptotic properties of consensus-type algorithms for networked systems with regime-switching topologies
- Coupling and ergodic theorems for Markov chains with damping component
- Asymptotic expansions for power-exponential moments of hitting times for nonlinearly perturbed semi-Markov processes
- How does a stochastic optimization/approximation algorithm adapt to a randomly evolving optimum/root with jump Markov sample paths
- A fluid-diffusion-hybrid limiting approximation for priority systems with fast and slow customers
- Discrete-time Markov chains with two-time scales and a countable state space: limit results and queueing applications
- Numerical methods for controlled regime-switching diffusions and regime-switching jump diffusions
- Integration by parts and martingale representation for a Markov chain
- Sequences of random matrices modulated by a discrete-time Markov chain*
- Analytic approximations of queues with lightly- and heavily-correlated autoregressive service times
- From random Poincaré maps to stochastic mixed-mode-oscillation patterns
- Fast computation of stationary joint probability distribution of sparse Markov chains
- Foundations of average-cost nonhomogeneous controlled Markov chains
- A novel HMM-based learning framework for improving dynamic wireless push system performance
- Stochastic optimization algorithms for barrier dividend strategies
- Stochastic systems arising from Markov modulated empirical measures
- Persistent tracking and identification of regime-switching systems with structural uncertainties: unmodeled dynamics, observation bias, and nonlinear model mismatch
- Adaptive stepsize selection for tracking in a regime-switching environment
- Markov chains and stochastic stability
- Discrete-time linear skew-Gaussian system and its recursive fixed-dimensional exact density filtering
- Asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes. I.
- On the notion of weak stability and related issues of hybrid diffusion systems
- A stochastic approximation algorithm for option pricing model calibration with a switchable market
This page was built for publication: Discrete-Time Markov Chains
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4831226)
