Markov Chains

DOI10.1007/978-1-4757-3124-8zbMath0949.60009OpenAlexW4245558686MaRDI QIDQ4256220

Publication date: 6 July 1999

Published in: Texts in Applied Mathematics (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1007/978-1-4757-3124-8

Mathematics Subject Classification ID

Random fields (60G60) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10) Applications of renewal theory (reliability, demand theory, etc.) (60K10) Continuous-time Markov processes on discrete state spaces (60J27) Research exposition (monographs, survey articles) pertaining to probability theory (60-02)

Related Items

Time-headway distribution for periodic totally asymmetric exclusion process with various updates, Hyperparameter estimation and plug-in kernel density estimates for maximum a posteriori land-cover classification with multiband satellite data, Catalytic branching random walk with semi-exponential increments, Moments of Markovian growth–collapse processes, Propagation of extrinsic fluctuations in biochemical birth-death processes, Randomized and Relaxed Strategies in Continuous-Time Markov Decision Processes, Learning management knowledge for manufacturing systems in the early stages using time series data, Strong Convergence for Split-Step Methods in Stochastic Jump Kinetics, Error Bounds for Augmented Truncations of Discrete-Time Block-Monotone Markov Chains under Subgeometric Drift Conditions, Stability and Strong Convergence for Spatial Stochastic Kinetics, Frog models on trees through renewal theory, The compound Poisson immigration process subject to binomial catastrophes, On-Line Process Control using Attributes with Misclassification Errors: An Economical Design for Short-Run Production, Generalized gambler's ruin problem: explicit formulas via Siegmund duality, Some sufficient conditions for stochastic comparisons between hitting times for skip-free Markov chains, Risk-sensitive stopping problems for continuous-time Markov chains, Piecewise linear process with renewal starting points, First crossing times of telegraph processes with jumps, Novel advancements in the Markov chain stock model: analysis and inference, Multi-type branching models to describe cell differentiation programs, How well-connected is the surface of the global ocean?, Learning Markov Models Via Low-Rank Optimization, Stability of Parallel Server Systems, On balance relations for irreversible chemical reaction networks, Stationary Averaging for Multiscale Continuous Time Markov Chains Using Parallel Replica Dynamics, Pathwise Error Bounds in Multiscale Variable Splitting Methods for Spatial Stochastic Kinetics, Propagation of chaos and moderate interaction for a piecewise deterministic system of geometrically enriched particles, Steady-state analysis of a multiserver queue in the Halfin-Whitt regime, On the Ergodicity of Slow-Varying Nonstationary Markov Chains, Simplicial gauge theory on spacetime, A Markov chain on the solution space of edge colorings of bipartite graphs, Approximation of the invariant measure with an Euler scheme for stochastic PDEs driven by space-time white noise, On classes of Bitcoin-inspired infinite-server queueing systems, Equilibrium and non-equilibrium Ising models by means of PCA, Systems of Ergodic BSDEs Arising in Regime Switching Forward Performance Processes, Bayesian selection probability estimation for probabilistic Boolean networks, Pinning dynamic systems of networks with Markovian switching couplings and controller-node set, Optimal search efficiency of Barker's algorithm with an exponential fitness function, Usual and stochastic tail orders between hitting times for two Markov chains, Maximum of catalytic branching random walk with regularly varying tails, Trajectory composition of Poisson time changes and Markov counting systems, Hopf algebras and Markov chains: two examples and a theory, Spectra of generalized stochastic adding machines, A perturbative probabilistic approach to quantum many-body systems, Convergence of value functions for finite horizon Markov decision processes with constraints, Stationary Distributions of Continuous-Time Markov Chains: A Review of Theory and Truncation-Based Approximations, Coexistence of localized Gibbs measures and delocalized gradient Gibbs measures on trees, Stability and optimality of a multi-product production and storage system under demand uncertainty, UTILITY MAXIMIZATION WITH INTERMEDIATE CONSUMPTION UNDER RESTRICTED INFORMATION FOR JUMP MARKET MODELS, Stationary flows and uniqueness of invariant measures, Probabilistic verification and approximation, An ergodic sampling scheme for constrained Hamiltonian systems with applications to molecular dynamics, Induction: From Kolmogorov and Solomonoff to De Finetti and Back to Kolmogorov, Sample path optimality for a Markov optimization problem, A semi‐algebraic approach that enables the design of inter‐grid operators to optimize multigrid convergence, Zero-automatic queues and product form, The Continuous-Time Ehrenfest Process in Term Structure Modelling, A queueing analysis of max-min fairness, proportional fairness and balanced fairness, Variational image inpainting, Testing for parameter stability in $RCA(1)$ time series, The order book as a queueing system: average depth and influence of the size of limit orders, Sojourn time distributions in the queue defined by a general QBD process, 𝑊-Markov measures, transfer operators, wavelets and multiresolutions, A sequential update algorithm for computing the stationary distribution vector in upper block-Hessenberg Markov chains, Fully asynchronous behavior of double-quiescent elementary cellular automata, Phase Transitions for Controlled Markov Chains on Infinite Graphs, Networked gain-scheduled fault diagnosis under control input dropouts without data delivery acknowledgment, Asymptotics of the transition probabilities of the simple random walk on self-similar graphs, Security and trust for wireless autonomic networks. Systems and control methods, On the probabilistic approach to heterogeneous structure interactions in agent-based computational models, A random cloud model for the Schrödinger equation, Multiple testing for neuroimaging via hidden Markov random field, Exponential Random Graphs as Models of Overlay Networks, Noise-Induced Resonance in Bistable Systems Caused by Delay Feedback, The Markov chain Monte Carlo revolution, Bounds in total variation distance for discrete-time processes on the sequence space, Estimate of the Interaction Neighborhood Radius for a Markov Random Field, A weak convergence criterion for constructing changes of measure, Bounding the Bias of Contrastive Divergence Learning, Stationary Distributions of Discrete-Time Markov Chains in Random Environment: Exact Computations and Bounds, Robustness of Markov processes on large networks, Competing first passage percolation on random regular graphs, Fcfs infinite bipartite matching of servers and customers, Analysis of a non-Markovian queueing model: Bayesian statistics and MCMC methods, Modified single sweep method for reconstructing free-energy landscapes, Hypergroups and distance distributions of random walks on graphs, Stability of the Bipartite Matching Model, On the Approximation of Transport Phenomena - a Dynamical Systems Approach, On the spectral radius and stiffness of Markov jump process rate matrices, Finite-Time Performance of Distributed Temporal-Difference Learning with Linear Function Approximation, STOCHASTIC ATTRACTORS FOR NON-ERGODIC MARKOV PROCESSES: SOME EXAMPLES, Factorization Identities for Reflected Processes, with Applications, Probability distributions of extinction times, species richness, and immigration and extinction rates in neutral ecological models, A stochastic matching model on hypergraphs, Co-jumps and Markov Counting Systems in Random Environments, Resolving Ambiguities in Reconstructed Grain Maps using Discrete Tomography, Approximate analysis of biological systems by hybrid switching jump diffusion, Sampling and Statistical Physics via Symmetry, The shape of a random affine Weyl group element and random core partitions, Long term behaviour of locally interacting birth-and-death processes, On the study of the running maximum and minimum level of level-dependent quasi-birth–death processes and related models, Analysis of Relaxation Time in Random Walk with Jumps, Semantic security for quantum wiretap channels, Markov evolution algebras, Pruned Discrete Random Samples, Doob: A Half-Century on, A Markovian arrival stream approach to stochastic gene expression in cells, A de-singularized meshfree approach to default probability estimation under a regime-switching synchronous-jump tempered stable Lévy model, Hitting time expressions for quantum channels: beyond the irreducible case and applications to unitary walks, Approximately counting and sampling knowledge states, COUNTABLE STATE MARKOV PROCESSES: NON-EXPLOSIVENESS AND MOMENT FUNCTION, On the Mathematical Consequences of Binning Spike Trains, A TWO-STATE MODEL FOR NOISE-INDUCED RESONANCE IN BISTABLE SYSTEMS WITH DELAY, A constrained hybrid Monte‐Carlo algorithm and the problem of calculating the free energy in several variables, LAPPED TRANSFORMS AND HIDDEN MARKOV MODELS FOR SEISMIC DATA FILTERING, Rarity and exponentiality: an extension of Keilson's theorem, with applications, Production-inventory systems with imperfect advance demand information and updating, Analysis of Petri Net Models through Stochastic Differential Equations, Binomial-Poisson entropic inequalities and the M/M/∞queue, Applications of Page Ranking in P Systems, Generalized Pólya urn Designs with Null Balance, Projektowanie systemów inżynierskich w polskich kopalniach w trzecim kwartale XX wieku, Fluctuations of the Propagation Front of a Catalytic Branching Walk, Homogenization of a Random Walk on a Graph in $\mathbb R^d$: An Approach to Predict Macroscale Diffusivity in Media with Finescale Obstructions and Interactions, Strategic experimentation in queues, Metastability in Markov processes, On Sloane’s Persistence Problem, Error Bounds for Augmented Truncations of Discrete-Time Block-Monotone Markov Chains under Geometric Drift Conditions, Strong Law of Large Numbers and Central Limit Theorems for Functionals of Inhomogeneous Semi-Markov Processes, Discrete-time classical and quantum Markovian evolutions: Maximum entropy problems on path space, Uniform Chernoff and Dvoretzky-Kiefer-Wolfowitz-type inequalities for Markov chains and related processes, Moment growth bounds on continuous time Markov processes on non-negative integer lattices, Distributed link scheduling in wireless networks, Random Walks, Directed Cycles, and Markov Chains, Phase transitions in the Ising model on a hierarchical random graph based on the triangle, The mixture transition distribution model for high-order Markov chains and non-Gaussian time series, Random walks systems with finite lifetime on $\mathbb {Z}$, Estimation of steady-state quantities of an HMM with some rarely generated emissions, Optimal inventory control with path-dependent cost criteria, A Bayesian beta Markov random field calibration of the term structure of implied risk neutral densities, The Wright-Fisher site frequency spectrum as a perturbation of the coalescent's, Sequential stratified regeneration: \textit{MCMC} for large state spaces with an application to subgraph count estimation, Asymptotic periodicity of recurrent flows in infinite networks, First hitting time of a high level by a catalytic branching walk, Efficient search and responsiveness trade-offs in a Markov chain model of evolution in dynamic environments, Dynamical criticality in gene regulatory networks, Spread of a catalytic branching random walk on a multidimensional lattice, Almost-invariant sets and invariant manifolds - connecting probabilistic and geometric descriptions of coherent structures in flows, A Bayesian hierarchical spatial model for dental caries assessment using non-Gaussian Markov random fields, Solution of linear ill-posed problems by model selection and aggregation, Stability in the self-organized evolution of networks, When are stochastic transition systems tameable?, Mean hitting time formula for positive maps, Gibbs sampler-based coordination of autonomous swarms, Random talk: Random walk and synchronizability in a moving neighborhood network, Continuous-time block-monotone Markov chains and their block-augmented truncations, Low-rank separated representation surrogates of high-dimensional stochastic functions: application in Bayesian inference, On the convergence rate of random permutation sampler and ECR algorithm in missing data models, Fuzzy spectral clustering by PCCA+: application to Markov state models and data classification, Comparison principles and applications to mathematical modelling of vegetal meta-communities, Monte Carlo Markov chains constrained on graphs for a target with disconnected support, Cycles and communicating classes in membrane systems and molecular dynamics, On time-reversal and space-time harmonic processes for Markovian quantum channels, A necessary and sufficient condition for diagnosability of stochastic discrete event systems, Spread of (mis)information in social networks, Monetary shocks in a spatial overlapping generations model, Verification of detectability in probabilistic finite automata, Orthogonal series estimates on strong spatial mixing data, Geometric ergodicity and the spectral gap of non-reversible Markov chains, Hitting times with taboo for a random walk, The flip-the-state transition operator for restricted Boltzmann machines, Mathematical modeling of distributed catastrophic and terrorist risks, Open quantum random walks on the half-line: the Karlin-McGregor formula, path counting and Foster's theorem, Detectability in stochastic discrete event systems, Common learning with intertemporal dependence, Training restricted Boltzmann machines: an introduction, Fast convergence in evolutionary models: a Lyapunov approach, Random walk distances in data clustering and applications, Spectral and graph-theoretic bounds on steady-state-probability estimation performance for an ergodic Markov chain, Strong and weak orders in averaging for SPDEs, Pattern hit-and-run for sampling efficiently on polytopes, On the infinitesimal dispersion of multivariate Markov counting systems, Maximum Kolmogorov-Sinai entropy versus minimum mixing time in Markov chains, Computing the average parallelism in trace monoids., Join the shortest queue among $k$ parallel queues: tail asymptotics of its stationary distribution, The cellular basis of cell sorting kinetics, Mixing times for the mean-field Blume-Capel model via aggregate path coupling, How small are small mutation rates?, A multi-cluster time aggregation approach for Markov chains, Modeling hybrid network dynamics under random perturbations, Entanglement in the quantum Ising model, Efficiency of simulation in monotone hyper-stable queueing networks, Around probabilistic cellular automata, $M/M/\infty$ queues in semi-Markovian random environment, Turing instability in a model with two interacting Ising lines: non-equilibrium fluctuations, Analytical evaluation of the output variability in production systems with general Markovian structure, Multiple pattern matching: a Markov chain approach, Modeling and inference for infectious disease dynamics: a likelihood-based approach, Uniform and Bernoulli measures on the boundary of trace monoids, A bound for the convergence rate of parallel tempering for sampling restricted Boltzmann machines, Open quantum random walks: ergodicity, hitting times, gambler's ruin and potential theory, On randomised strategies in the $\lambda $-calculus, A sufficient condition for the subexponential asymptotics of GI/G/$1$-type Markov chains with queueing applications, Binary Markov random fields and interpretable mass spectra discrimination, Clustering in block Markov chains, A probabilistic interpretation of the Macdonald polynomials, A martingale decomposition of discrete Markov chains, Change point dynamics for financial data: an indexed Markov chain approach, Statistical estimation with bounded memory, High-dimensional structure estimation in Ising models: local separation criterion, Zero-automatic networks, An asymptotic maximum principle for essentially linear evolution models, Chaos, spatial extension, transport, and nonequilibrium thermodynamics, Random walks on the BMW monoid: an algebraic approach, Consistent micro, macro and state-based population modelling, Realizable monotonicity for continuous-time Markov processes, Compound Markov counting processes and their applications to modeling infinitesimally over-dispersed systems, Time operator of Markov chains and mixing times. Applications to financial data, Pursuit of food \textit{versus} pursuit of information in a Markovian perception-action loop model of foraging, Stochastic adding machines based on Bratteli diagrams, Safe navigation in adversarial environments, Ricci curvature of Markov chains on metric spaces, The adjusted Viterbi training for hidden Markov models, Mean reversal for stochastic hybrid systems, Improving efficiency of data augmentation algorithms using Peskun's theorem, Effect of energy-saving server scheduling on power consumption for large-scale data centers, Time series analysis via mechanistic models, Hierarchical annealing for synthesis of binary images, A Monte Carlo method for filtering a marked doubly stochastic Poisson process, A note on the attractor-property of infinite-state Markov chains, Signaling for decentralized routing in a queueing network, Distributed computation with continual population growth, A direct solution to the stochastic inverse eigenvalue problem for complex-valued eigenspectra, Generalized Markov chain tree theorem and Kemeny's constant for a class of non-Markovian matrices, Collective proposal distributions for nonlinear MCMC samplers: mean-field theory and fast implementation