Discrete-Time Markov Chains
From MaRDI portal
Publication:4831226
DOI10.1007/B138226zbMATH Open1072.60002OpenAlexW4299639519MaRDI QIDQ4831226FDOQ4831226
Publication date: 22 December 2004
Published in: Stochastic Modelling and Applied Probability (Search for Journal in Brave)
Full work available at URL: https://doi.org/10.1007/b138226
Recommendations
- Continuous-time Markov chains and applications. A two-time-scale approach
- Markov Processes and Applications
- Markov chains and stochastic stability
- Continuous-time Markov decision processes. Borel space models and general control strategies. With a foreword by Albert Nikolaevich Shiryaev
- Markov chains. Theory and applications
Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) (60J20) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10) Dynamic programming (90C39) Research exposition (monographs, survey articles) pertaining to probability theory (60-02) Research exposition (monographs, survey articles) pertaining to operations research and mathematical programming (90-02) Production models (90B30) Markov and semi-Markov decision processes (90C40) Optimal stochastic control (93E20)
Cited In (50)
- Stability of Discrete-Time Regime-Switching Dynamic Systems with Delays
- Singular perturbation for the discounted continuous control of piecewise deterministic Markov processes
- Distribution of PageRank Mass Among Principle Components of the Web
- Asymptotic optimality for consensus-type stochastic approximation algorithms using iterate averaging
- Asymptotic properties of Markov-modulated random sequences with fast and slow timescales
- Flows of rare events for regularly perturbed semi-Markov processes
- Continuous-time Markov chains and applications. A two-time-scale approach
- Asymptotic properties of hybrid random processes modulated by Markov chains
- System identification: regime switching, unmodeled dynamics, and binary sensors
- Bounding the lumping error in Markov chain dynamics
- Multiple time scale decomposition of discrete time Markov chains
- Pathwise convergence rates for numerical solutions of Markovian switching stochastic differential equations
- Numerical solutions for jump-diffusions with regime switching
- Dissipative control of a three-species food chain stochastic system with a hidden Markov chain
- Optimal stopping of Markov switching Lévy processes
- Discrete-time hybrid control in Borel spaces
- Perturbed Markov chains with damping component
- Ergodic theory for controlled Markov chains with stationary inputs
- Discrete-time semi-Markov random evolutions -- average and diffusion approximation of difference equations and additive functionals
- Tracking and identification of regime-switching systems using binary sensors
- Safety verification for regime-switching jump diffusions via barrier certificates
- Stochastic competitive Lotka-Volterra ecosystems under partial observation: feedback controls for permanence and extinction
- Balanced realizations of regime-switching linear systems
- Weak convergence of Markov-modulated random sequences
- Discrete-time semi-Markov random evolutions and their applications
- Coupling and ergodic theorems for Markov chains with damping component
- Asymptotic properties of consensus-type algorithms for networked systems with regime-switching topologies
- Option pricing in a regime-switching model using the fast Fourier transform
- Asymptotic expansions for power-exponential moments of hitting times for nonlinearly perturbed semi-Markov processes
- How does a stochastic optimization/approximation algorithm adapt to a randomly evolving optimum/root with jump Markov sample paths
- A fluid-diffusion-hybrid limiting approximation for priority systems with fast and slow customers
- Discrete-time Markov chains with two-time scales and a countable state space: limit results and queueing applications
- Numerical methods for controlled regime-switching diffusions and regime-switching jump diffusions
- Sequences of random matrices modulated by a discrete-time Markov chain*
- Integration by parts and martingale representation for a Markov chain
- Analytic approximations of queues with lightly- and heavily-correlated autoregressive service times
- From random Poincaré maps to stochastic mixed-mode-oscillation patterns
- Fast computation of stationary joint probability distribution of sparse Markov chains
- Foundations of average-cost nonhomogeneous controlled Markov chains
- A novel HMM-based learning framework for improving dynamic wireless push system performance
- Stochastic systems arising from Markov modulated empirical measures
- Stochastic optimization algorithms for barrier dividend strategies
- Persistent tracking and identification of regime-switching systems with structural uncertainties: unmodeled dynamics, observation bias, and nonlinear model mismatch
- Adaptive stepsize selection for tracking in a regime-switching environment
- Discrete-time linear skew-Gaussian system and its recursive fixed-dimensional exact density filtering
- Markov chains and stochastic stability
- Asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes. I.
- On the notion of weak stability and related issues of hybrid diffusion systems
- A stochastic approximation algorithm for option pricing model calibration with a switchable market
- Convergence of Markov chain approximation on generalized HJB equation and its applications
This page was built for publication: Discrete-Time Markov Chains
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4831226)
