Discrete-Time Markov Chains
From MaRDI portal
Research exposition (monographs, survey articles) pertaining to probability theory (60-02) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10) Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) (60J20) Research exposition (monographs, survey articles) pertaining to operations research and mathematical programming (90-02) Production models (90B30) Dynamic programming (90C39) Markov and semi-Markov decision processes (90C40) Optimal stochastic control (93E20)
Recommendations
- Continuous-time Markov chains and applications. A two-time-scale approach
- Markov Processes and Applications
- Markov chains and stochastic stability
- Continuous-time Markov decision processes. Borel space models and general control strategies. With a foreword by Albert Nikolaevich Shiryaev
- Markov chains. Theory and applications
Cited in
(52)- How does a stochastic optimization/approximation algorithm adapt to a randomly evolving optimum/root with jump Markov sample paths
- Tracking and identification of regime-switching systems using binary sensors
- Bounding the lumping error in Markov chain dynamics
- Multiple time scale decomposition of discrete time Markov chains
- Dissipative control of a three-species food chain stochastic system with a hidden Markov chain
- Fast computation of stationary joint probability distribution of sparse Markov chains
- Stochastic systems arising from Markov modulated empirical measures
- Integration by parts and martingale representation for a Markov chain
- Ergodic theory for controlled Markov chains with stationary inputs
- Analytic approximations of queues with lightly- and heavily-correlated autoregressive service times
- Discrete-time hybrid control in Borel spaces
- Perturbed Markov chains with damping component
- Asymptotic properties of consensus-type algorithms for networked systems with regime-switching topologies
- Asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes. I.
- From random Poincaré maps to stochastic mixed-mode-oscillation patterns
- Convergence of Markov chain approximation on generalized HJB equation and its applications
- Adaptive stepsize selection for tracking in a regime-switching environment
- On the notion of weak stability and related issues of hybrid diffusion systems
- Persistent tracking and identification of regime-switching systems with structural uncertainties: unmodeled dynamics, observation bias, and nonlinear model mismatch
- Optimal stopping of Markov switching Lévy processes
- Discrete-time semi-Markov random evolutions -- average and diffusion approximation of difference equations and additive functionals
- A stochastic approximation algorithm for option pricing model calibration with a switchable market
- Asymptotic properties of Markov-modulated random sequences with fast and slow timescales
- Weak convergence of Markov-modulated random sequences
- Asymptotic expansions for power-exponential moments of hitting times for nonlinearly perturbed semi-Markov processes
- Coupling and ergodic theorems for Markov chains with damping component
- Discrete-time Markov chains with two-time scales and a countable state space: limit results and queueing applications
- Stability of Discrete-Time Regime-Switching Dynamic Systems with Delays
- Stochastic competitive Lotka-Volterra ecosystems under partial observation: feedback controls for permanence and extinction
- System identification: regime switching, unmodeled dynamics, and binary sensors
- Asymptotic properties of hybrid random processes modulated by Markov chains
- Continuous-time Markov chains and applications. A two-time-scale approach
- Pathwise convergence rates for numerical solutions of Markovian switching stochastic differential equations
- Discrete-time semi-Markov random evolutions and their applications
- Foundations of average-cost nonhomogeneous controlled Markov chains
- A fluid-diffusion-hybrid limiting approximation for priority systems with fast and slow customers
- Singular perturbation for the discounted continuous control of piecewise deterministic Markov processes
- Asymptotic optimality for consensus-type stochastic approximation algorithms using iterate averaging
- Distribution of PageRank Mass Among Principle Components of the Web
- Numerical solutions for jump-diffusions with regime switching
- Safety verification for regime-switching jump diffusions via barrier certificates
- Sequences of random matrices modulated by a discrete-time Markov chain*
- Flows of rare events for regularly perturbed semi-Markov processes
- A novel HMM-based learning framework for improving dynamic wireless push system performance
- Discrete-time linear skew-Gaussian system and its recursive fixed-dimensional exact density filtering
- Markov chains and stochastic stability
- Reducing metastable continuous-space Markov chains to Markov chains on a finite set
- A kernel-based stochastic approximation framework for contextual optimization
- Numerical methods for controlled regime-switching diffusions and regime-switching jump diffusions
- Option pricing in a regime-switching model using the fast Fourier transform
- Stochastic optimization algorithms for barrier dividend strategies
- Balanced realizations of regime-switching linear systems
This page was built for publication: Discrete-Time Markov Chains
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4831226)
