Markov Chains

DOI10.1017/CBO9780511810633zbMath0938.60058OpenAlexW755194005WikidataQ56769020 ScholiaQ56769020MaRDI QIDQ4216130

Publication date: 20 October 1998

Full work available at URL: https://doi.org/10.1017/cbo9780511810633

Introductory exposition (textbooks, tutorial papers, etc.) pertaining to probability theory (60-01) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10)

Related Items

The generalized distance spectrum of a graph and applications ⋮ Stochastic algorithms for self-consistent calculations of electronic structures ⋮ Keeping random walks safe from extinction and overpopulation in the presence of life-taking disasters ⋮ Arcsine and Darling–Kac laws for piecewise linear random interval maps ⋮ Classical and Quantum Random-Walk Centrality Measures in Multilayer Networks ⋮ Structural classification of continuous time Markov chains with applications ⋮ Robust and consistent estimation of generators in credit risk ⋮ Optimal Control of Continuous-Time Markov Chains with Noise-Free Observation ⋮ Some harmonic functions for killed Markov branching processes with immigration and culling ⋮ Optimal Liquidity-Based Trading Tactics ⋮ Quantifying transient spreading dynamics on networks ⋮ Inference on autoregulation in gene expression with variance-to-mean ratio ⋮ Slowest first passage times, redundancy, and menopause timing ⋮ Lines of descent in a Moran model with frequency-dependent selection and mutation ⋮ Expansion and flooding in dynamic random networks with node churn ⋮ Aggregation Methods for Computing Steady States in Statistical Physics ⋮ Markov chains under nonlinear expectation ⋮ Almost Sure and Moment Exponential Stability of Regime-Switching Jump Diffusions ⋮ Transaction time models in multi-state life insurance ⋮ How well-connected is the surface of the global ocean? ⋮ Fitting feature-dependent Markov chains ⋮ Signed network representation with novel node proximity evaluation ⋮ Queuing models with Mittag-Leffler inter-event times ⋮ A greedy chip‐firing game ⋮ A parallel variable neighborhood search approach for the obnoxious p‐median problem ⋮ Asymptotic decomposition of substochastic semigroups and applications ⋮ Deep learning's shallow gains: a comparative evaluation of algorithms for automatic music generation ⋮ The scaling limit of a critical random directed graph ⋮ Sensitivity of Steady States in Networks with Application to Markov Chains and Chemical Reaction Networks ⋮ Random amenable C^*-algebras ⋮ Delta-Bose gas from the viewpoint of the two-dimensional stochastic heat equation ⋮ Scaling limit of an adaptive contact process ⋮ Unnamed Item ⋮ Stochastic filtering and optimal control of pure jump Markov processes with noise-free partial observation ⋮ Approximately counting and sampling knowledge states ⋮ Quasi-equilibria and click times for a variant of Muller's ratchet ⋮ Almost triangular Markov chains on \(\mathbb{N}\) ⋮ On the estimation of partially observed continuous-time Markov chains ⋮ Harnack inequality and one-endedness of UST on reversible random graphs ⋮ Markovian persuasion with two states ⋮ A combinatorial representation for the invariant measure of diffusion processes on metric graphs ⋮ DIFFUSION LIMITS FOR A MARKOV MODULATED BINOMIAL COUNTING PROCESS ⋮ Full classification of dynamics for one-dimensional continuous-time Markov chains with polynomial transition rates ⋮ Fluctuation bounds for continuous time branching processes and evolution of growing trees with a change point ⋮ On the φ-Selmer groups of the elliptic curvesy²=x³−Dx ⋮ Asymptotic and Numerical Analysis of a Stochastic PDE Model of Volume Transmission ⋮ Unnamed Item ⋮ Opinion Manipulation in Social Networks ⋮ Sequential Noise-Induced Escapes for Oscillatory Network Dynamics ⋮ A Probabilistic Analysis of Volume Transmission in the Brain ⋮ Unnamed Item ⋮ Unnamed Item ⋮ A Transition to Sharp Timing in Stochastic Leaky Integrate-and-Fire Neurons Driven by Frozen Noisy Input ⋮ Concentration of quantum states from quantum functional and transportation cost inequalities ⋮ Usual and stochastic tail orders between hitting times for two Markov chains ⋮ Decay property of stopped Markovian bulk-arriving queues ⋮ Longitudinal data analysis of mean passage time among malnutrition states: an application of Markov chains ⋮ An extended likelihood framework for modelling discretely observed credit rating transitions ⋮ Random walk Green kernels in the neutral Moran model conditioned on survivors at a random time to origin ⋮ Quantum mixing of Markov chains for special distributions ⋮ Fast Non-mean-field Networks: Uniform in Time Averaging ⋮ Learning-Based Mean-Payoff Optimization in an Unknown MDP under Omega-Regular Constraints ⋮ Model Decomposition and Stochastic Fragments ⋮ Continuum Approximation of Invasion Probabilities ⋮ Ideal flow of Markov Chain ⋮ Geometric fluid approximation for general continuous-time Markov chains ⋮ Equilibrium and termination II: the case of Petri nets ⋮ On the exponential stability of switching-diffusion processes with jumps ⋮ Generalized product-form stationary distributions for Markov chains in random environments with queueing applications ⋮ A reversible allelic partition process and Pitman sampling formula ⋮ 𝑊-Markov measures, transfer operators, wavelets and multiresolutions ⋮ Graphically Balanced Equilibria and Stationary Measures of Reaction Networks ⋮ Analysis of Nonprocessive Molecular Motor Transport Using Renewal Reward Theory ⋮ Analytic-geometric methods for finite Markov chains with applications to quasi-stationarity ⋮ Fluctuations in a general preferential attachment model via Stein's method ⋮ Data Uncertainty in Markov Chains: Application to Cost-Effectiveness Analyses of Medical Innovations ⋮ General Contact Process with Rapid Stirring ⋮ Symbolic computation of differential equivalences ⋮ Behaviour of the Gibbs sampler when conditional distributions are potentially incompatible ⋮ M-estimator and its weak consistency for a (2, 1) random walk in a parametric random environment ⋮ Cluster-based feedback control of turbulent post-stall separated flows ⋮ Mapping Return Values of Extreme Wind Speeds ⋮ User-Optimal State-Dependent Routeing in Parallel Tandem Queues with Loss ⋮ Self-Sustaining Iterated Learning ⋮ Bounded-Size Rules: The Barely Subcritical Regime ⋮ Markov loops in discrete spaces ⋮ Coarse-Graining Open Markov Processes ⋮ Synchronisation conditions in the Kuramoto model and their relationship to seminorms ⋮ Tail of a linear diffusion with Markov switching ⋮ Multiscale Modeling of Wear Degradation in Cylinder Liners ⋮ A General Framework for Consistent Estimation of Charge Transport Properties via Random Walks in Random Environments ⋮ Unnamed Item ⋮ A generating function approach to Markov chains undergoing binomial catastrophes ⋮ Probabilistic Foundations of Spatial Mean-Field Models in Ecology and Applications ⋮ Interacting Particle Systems: Hydrodynamic Limit Versus High Density Limit ⋮ Reaction-Subdiffusion Equations with Species-Dependent Movement ⋮ Sharp Entrywise Perturbation Bounds for Markov Chains ⋮ The Buck-Passing Game ⋮ Adversarial Patrolling in a Uniform ⋮ Quantum walk on the line through potential barriers ⋮ Complex systems: features, similarity and connectivity ⋮ Simple SIR models with Markovian control ⋮ Lumpability for uncertain continuous-time Markov chains ⋮ A Feynman-Kac-Itô formula for magnetic Schrödinger operators on graphs ⋮ Sampling-rate-dependent probabilistic Boolean networks ⋮ Elucidation of T cell signalling models ⋮ Markov chain approximation of pure jump processes ⋮ Derivatives of the stochastic growth rate ⋮ Gray codes generation algorithm and theoretical evaluation of random walks in \(N\)-cubes ⋮ The probability of epidemic fade-out is non-monotonic in transmission rate for the Markovian SIR model with demography ⋮ Reversible polymorphism-aware phylogenetic models and their application to tree inference ⋮ A computationally grounded, weighted doxastic logic ⋮ On extinction time of a generalized endemic chain-binomial model ⋮ Positivity and contractivity in the dynamics of clusters' splitting with derivative of fractional order ⋮ Duplication in a model of rock fracture with fractional derivative without singular kernel ⋮ On the equivalence between SUE and fixed-point states of day-to-day assignment processes with serially-correlated route choice ⋮ Random walks on binary strings applied to the somatic hypermutation of B-cells ⋮ Central limit theorem through expansion of the propagation of chaos for Bird and Nanbu systems ⋮ Trading performance for stability in Markov decision processes ⋮ Extending the multi-level method for the simulation of stochastic biological systems ⋮ An exact discretization of a Lax equation for shock clustering and Burgers turbulence. I: Dynamical aspects and exact solvability ⋮ Stability of stochastic semigroups and applications to Stein's neuronal model ⋮ Hitting time for Bessel processes-walk on moving spheres algorithm (WoMS) ⋮ Additional aspects of the generalized linear-fractional branching process ⋮ Semi-Markov models and motion in heterogeneous media ⋮ A computational model of outguessing in two-player non-cooperative games ⋮ The stochastic fluid-fluid model: a stochastic fluid model driven by an uncountable-state process, which is a stochastic fluid model itself ⋮ Explosion, implosion, and moments of passage times for continuous-time Markov chains: a semimartingale approach ⋮ Temperature in and out of equilibrium: a review of concepts, tools and attempts ⋮ A mathematical analysis of fractional fragmentation dynamics with growth ⋮ Markov chain aggregation and its applications to combinatorial reaction networks ⋮ On Mittag-Leffler distributions and related stochastic processes ⋮ A scaling analysis of a cat and mouse Markov chain ⋮ Batch arrival single-server queue with variable service speed and setup time ⋮ Synchronization of reinforced stochastic processes with a network-based interaction ⋮ Discrete fractal dimensions of the ranges of random walks in \(\mathbb Z^d\) associate with random conductances ⋮ Tracking human pose with multiple activity models ⋮ Markov fundamental tensor and its applications to network analysis ⋮ A general framework for never-ending learning from time series streams ⋮ Infinite-step opacity and \(K\)-step opacity of stochastic discrete-event systems ⋮ Existence results for a Michaud fractional, nonlocal, and randomly position structured fragmentation model ⋮ Stochastic interacting particle systems and nonlinear kinetic equations ⋮ Linear autonomous compartmental models as continuous-time Markov chains: transit-time and age distributions ⋮ Explicit solutions to dynamic diffusion-type equations and their time integrals ⋮ On the speed of a cookie random walk ⋮ Limit theorems for Markov processes indexed by continuous time Galton-Watson trees ⋮ Markov-modulated infinite-server queues with general service times ⋮ Tail asymptotics of a Markov-modulated infinite-server queue ⋮ A transformation of Markov jump processes and applications in genetic study ⋮ Approximation of stationary processes by hidden Markov models ⋮ On the mean residence time in stochastic lattice-gas models ⋮ Endogenous growth and global divergence in a multi-country agent-based model ⋮ The design and analysis of a generalized RESTART/DPR algorithm for rare event simulation ⋮ Penalization for birth and death processes ⋮ A probabilistic view on the deterministic mutation-selection equation: dynamics, equilibria, and ancestry via individual lines of descent ⋮ Maximum likelihood estimation of symmetric group-based models via numerical algebraic geometry ⋮ An extended Moran process that captures the struggle for fitness ⋮ Coalescence times for three genes provide sufficient information to distinguish population structure from population size changes ⋮ Bayesian parameter identification for Turing systems on stationary and evolving domains ⋮ Approximating time to extinction for endemic infection models ⋮ Transition choice probabilities in logit ⋮ Estimation of weak ARMA models with regime changes ⋮ The Wright-Fisher model with efficiency ⋮ Invariance, quasi-invariance, and unimodularity for random graphs ⋮ Stochastic transcription elongation via rule based modelling ⋮ Modified log-Sobolev inequality for a compact pure jump Markov process with degenerate jumps ⋮ Demographic noise slows down cycles of dominance ⋮ On randomised strategies in the \(\lambda \)-calculus ⋮ On site percolation in random quadrangulations of the half-plane ⋮ Non-explosivity of stochastically modeled reaction networks that are complex balanced ⋮ On the block counting process and the fixation line of the Bolthausen-Sznitman coalescent ⋮ Level 2.5 large deviations for continuous-time Markov chains with time periodic rates ⋮ Entropy production in nonlinear recombination models ⋮ Determining majority in networks with local interactions and very small local memory ⋮ Change point detection in network models: preferential attachment and long range dependence ⋮ Statistical analysis of the first passage path ensemble of jump processes ⋮ Average long-lived binary consensus: quantifying the stabilizing role played by memory ⋮ Labor migrant networks: growth, saturation, and deflection to new labor markets ⋮ Antibody-mediated immobilization of virions in mucus ⋮ The Cambridge Mathematical Journal and its descendants: the linchpin of a research community in the early and mid-Victorian age ⋮ Stability of nonlinear filters in nonmixing case ⋮ On strong Feller property, exponential ergodicity and large deviations principle for stochastic damping Hamiltonian systems with state-dependent switching ⋮ Extending transition path theory: periodically driven and finite-time dynamics ⋮ Monte Carlo methods in statistical physics: mathematical foundations and strategies ⋮ Asymptotic behavior of the rate of adaptation ⋮ On discrete-time semi-Markov processes ⋮ The dynamics of stochastic mono-molecular reaction systems in stochastic environments ⋮ Haldane linearisation done right: solving the nonlinear recombination equation the easy way ⋮ Remarks on the speeds of a class of random walks on the integers ⋮ A multi-time-scale analysis of chemical reaction networks. II: Stochastic systems ⋮ A stochastic differential equation SIS epidemic model with regime switching ⋮ Solving the selection-recombination equation: ancestral lines and dual processes ⋮ Explicit bounds for critical infection rates and expected extinction times of the contact process on finite random graphs ⋮ Recurrence criteria for generalized Dirichlet forms ⋮ How liquid is biological signalling? ⋮ Essential spectrum and Weyl asymptotics for discrete Laplacians ⋮ Twitter event networks and the superstar model ⋮ On normality in shifts of finite type ⋮ Diameter and stationary distribution of random \(r\)-out digraphs ⋮ On a cover time problem on a dynamic graph with steps at random times ⋮ Ergodicity of exclusion semigroups constructed from quantum Bernoulli noises ⋮ Individual Molecules Dynamics in Reaction Network Models ⋮ A countable-type branching process model for the tug-of-war cancer cell dynamics ⋮ Variational structures beyond gradient flows: a macroscopic fluctuation-theory perspective ⋮ Quantitative Coarse-Graining of Markov Chains ⋮ UNCERTAINTY VISUALIZATION FOR CHARACTERIZING HETEROGENEOUS HUMAN BEHAVIORS IN DISCRETE DYNAMICAL SYSTEM MODELS ⋮ Estimation of steady-state quantities of an HMM with some rarely generated emissions ⋮ A comparative analysis of knowledge acquisition performance in complex networks ⋮ Semi-Markovian capacities in production network models ⋮ A classification of the dynamics of three-dimensional stochastic ecological systems ⋮ A probabilistic approach to convex \((\phi)\)-entropy decay for Markov chains ⋮ Periodic words, common subsequences and frogs ⋮ Markov stochasticity coordinates ⋮ Noisy Hegselmann-Krause systems: phase transition and the \(2R\)-conjecture ⋮ Existence of the zero range process and a deposition model with superlinear growth rates ⋮ Grover search with lackadaisical quantum walks ⋮ A spectral decomposition for a simple mutation model ⋮ Computational Identification of Irreducible State-Spaces for Stochastic Reaction Networks ⋮ Model Checking Probabilistic Systems ⋮ A sojourn-based approach to semi-Markov reinforcement learning ⋮ \((\mathrm{BV},L^p)\)-decomposition, \(p = 1,2\), of functions in metric random walk spaces ⋮ First hitting time of Brownian motion on simple graph with skew semiaxes ⋮ General selection models: Bernstein duality and minimal ancestral structures ⋮ Root finding algorithms and persistence of Jordan centrality in growing random trees ⋮ Optimizing enzymatic catalysts for rapid turnover of substrates with low enzyme sequestration ⋮ Multi-competitive viruses over time-varying networks with mutations and human awareness ⋮ Bayesian calibration, validation and uncertainty quantification for predictive modelling of tumour growth: a tutorial ⋮ Passage times, exit times and Dirichlet problems for open quantum walks ⋮ A Probabilistic Analysis of the Reduction Ratio in the Suffix-Array IS-Algorithm ⋮ Mathematical representation of Clausius' and Kelvin's statements of the second law and irreversibility ⋮ No-oscillation theorem for the transient dynamics of the linear signal transduction pathway and beyond ⋮ Speed of excited random walks with long backward steps ⋮ Understanding autoencoders with information theoretic concepts ⋮ Relative submajorization and its use in quantum resource theories ⋮ A low-rank spectral method for learning Markov models ⋮ Stochastic dynamic analysis of nonlinear MDOF systems with chaotic motion under combined additive and multiplicative excitation ⋮ How fast can we reach a target vertex in stochastic temporal graphs? ⋮ Transition graph decomposition for complex balanced reaction networks with non-mass-action kinetics ⋮ Inferring density-dependent population dynamics mechanisms through rate disambiguation for logistic birth-death processes ⋮ Fractional random walk lattice dynamics ⋮ A remark on normalizations in a local large deviations principle for inhomogeneous birth-and-death process ⋮ Detailed balance \(=\) complex balance \(+\) cycle balance: a graph-theoretic proof for reaction networks and Markov chains ⋮ Always doing your best? Effort and performance in dynamic settings ⋮ A practical guide to mathematical methods for estimating infectious disease outbreak risks ⋮ Exact long time behavior of some regime switching stochastic processes ⋮ Area anomaly in the rough path Brownian scaling limit of hidden Markov walks ⋮ Maximum likelihood estimator consistency for a ballistic random walk in a parametric random environment ⋮ Upper escape rate of Markov chains on weighted graphs ⋮ Existence of geometric ergodic periodic measures of stochastic differential equations ⋮ On an irreducibility type condition for the ergodicity of nonconservative semigroups ⋮ The dynamics of luminal depletion and the stochastic gating of \(\operatorname{Ca}^{2 +} \)-activated \(\operatorname{Ca}^{2 +}\) channels and release sites ⋮ The Wang-Landau algorithm reaches the flat histogram criterion in finite time ⋮ Thermal stability of the Nagaoka-Thouless theorems ⋮ Local abstraction refinement for probabilistic timed programs ⋮ Particles Moving Around a Two-Lane Circular Track in Both Directions. Avoiding Collisions Leads to Self-Organization ⋮ Statistical inference for Markov chains with applications to credit risk ⋮ Tier structure of strongly endotactic reaction networks ⋮ Regenerative real trees ⋮ Robustness of community structure to node removal ⋮ Long lived transients in gene regulation ⋮ Evolution of discrete populations and the canonical diffusion of adaptive dynamics ⋮ Mathematical and numerical models for the cardiac electromechanical function ⋮ Stochastic precedence and minima among dependent variables ⋮ A continuous-time semi-Markov model for animal movement in a dynamic environment ⋮ Stationary flows and uniqueness of invariant measures ⋮ Unnamed Item ⋮ A predator-prey SIR type dynamics on large complete graphs with three phase transitions ⋮ Coined quantum walks lift the cospectrality of graphs and trees ⋮ Faster statistical model checking for unbounded temporal properties ⋮ Relaxation patterns and semi-Markov dynamics ⋮ Generalized Schrödinger semigroups on infinite graphs ⋮ Asymptotic behavior and aging of a low temperature cascading 2-GREM dynamics at extreme time scales ⋮ Optimizing the stable behavior of parameter-dependent dynamical systems -- maximal domains of attraction, minimal absorption times ⋮ Coevolutionary systems and PageRank ⋮ The general recombination equation in continuous time and its solution ⋮ Gradient flow formulations of discrete and continuous evolutionary models: a unifying perspective ⋮ Randomized construction of complexes with large diameter ⋮ An extension of the Moran process using type-specific connection graphs ⋮ Boundary Value Problems for Statistics of Diffusion in a Randomly Switching Environment: PDE and SDE Perspectives ⋮ Moment Semantics for Reversible Rule-Based Systems ⋮ Mean-Field Limits Beyond Ordinary Differential Equations ⋮ A local large deviation principle for inhomogeneous birth-death processes ⋮ Switching problems with controlled randomisation and associated obliquely reflected BSDEs ⋮ Novel dissipative properties of the master equation ⋮ Hidden Markov models: inverse filtering, belief estimation and privacy protection ⋮ Time consistent mean-variance asset allocation for a DC plan with regime switching under a jump-diffusion model ⋮ Link prediction techniques, applications, and performance: a survey ⋮ Tightness of Bernoulli Gibbsian line ensembles ⋮ Regional patterns in technological progress of Poland: the role of EU structural funds ⋮ Cutoff for polymer pinning dynamics in the repulsive phase ⋮ An enhanced model of a two player singled out game ⋮ Gaps and interleaving of point processes in sampling from a residual allocation model ⋮ Designing drug regimens that mitigate nonadherence ⋮ Decay rates for stabilization of linear continuous-time systems with random switching ⋮ Mixing time of Markov chains for the 1-2 model ⋮ Lines of descent in the deterministic mutation-selection model with pairwise interaction ⋮ Data-driven modeling of the temporal evolution of breakers' states in the French electrical transmission grid ⋮ Discrepancies between extinction events and boundary equilibria in reaction networks ⋮ On-diagonal lower estimate of heat kernels for locally finite graphs and its application to the semilinear heat equations ⋮ Explosion phenomena in stochastic coagulation-fragmentation models ⋮ One-dimensional Markov random fields, Markov chains and topological Markov fields ⋮ A biomathematical view on the fractional dynamics of cellulose degradation ⋮ The dynamics of fitness and wealth distributions -- a stochastic game-theoretic model ⋮ Block Markov chains on trees ⋮ Discrete-space partial dynamic equations on time scales and applications to stochastic processes ⋮ Normality in non-integer bases and polynomial time randomness