Markov chains and invariant probabilities

Related Items (65)

Time and Ratio Expected Average Cost Optimality for Semi-Markov Control Processes on Borel Spaces ⋮ Gradient flows in metric random walk spaces ⋮ Unnamed Item ⋮ Unnamed Item ⋮ Diffusion in large networks ⋮ A semidefinite optimization approach to the steady-state analysis of queueing systems ⋮ \((\mathrm{BV},L^p)\)-decomposition, \(p = 1,2\), of functions in metric random walk spaces ⋮ Finite-Memory Strategies in POMDPs with Long-Run Average Objectives ⋮ Perfect simulation of steady-state Markov chain on mixed state space ⋮ Geometry of information structures, strategic measures and associated stochastic control topologies ⋮ On Linear Programming for Constrained and Unconstrained Average-Cost Markov Decision Processes with Countable Action Spaces and Strictly Unbounded Costs ⋮ Nonparametric estimation of the jump rate for non-homogeneous marked renewal processes ⋮ On sets of occupational measures generated by a deterministic control system on an infinite time horizon ⋮ On Stochastic Stability of a Class of non-Markovian Processes and Applications in Quantization ⋮ Near optimality of quantized policies in stochastic control under weak continuity conditions ⋮ Bounding Stationary Averages of Polynomial Diffusions via Semidefinite Programming ⋮ Markov chain aggregation and its applications to combinatorial reaction networks ⋮ Simplest random walk for approximating Robin boundary value problems and ergodic limits of reflected diffusions ⋮ Estimation and testing for a Poisson autoregressive model ⋮ Representation theory and multilevel filters ⋮ Bootstrap, Markov chain Monte Carlo, and LP/SDP hierarchy for the lattice Ising model ⋮ Subcritical superprocesses conditioned on non-extinction ⋮ Layered networks, equilibrium dynamics, and stable coalitions ⋮ Correction to: ``Layered networks, equilibrium dynamics, and stable coalitions ⋮ Asymptotic Expansions for Stationary Distributions of Perturbed Semi-Markov Processes ⋮ An ergodic decomposition defined by regular jointly measurable Markov semigroups on Polish spaces ⋮ Average control of Markov decision processes with Feller transition probabilities and general action spaces ⋮ A Transition to Sharp Timing in Stochastic Leaky Integrate-and-Fire Neurons Driven by Frozen Noisy Input ⋮ Average Cost Optimality Inequality for Markov Decision Processes with Borel Spaces and Universally Measurable Policies ⋮ Unnamed Item ⋮ Ergodicity of observation-driven time series models and consistency of the maximum likelihood estimator ⋮ On nonlinear discrete-time systems driven by Markov chains ⋮ Decomposition formula and stationary measures for stochastic Lotka-Volterra system with applications to turbulent convection ⋮ Solutions of the average cost optimality equation for Markov decision processes with weakly continuous kernel: the fixed-point approach revisited ⋮ An ergodic decomposition defined by transition probabilities ⋮ Multidimensional random walk with reflections ⋮ Stochastic optimal control and linear programming approach ⋮ Iterated expectations, compact spaces, and common priors ⋮ Ergodicity and stability of a dynamical system perturbed by impulsive random interventions ⋮ On the ergodic decomposition for a class of Markov chains ⋮ Exploring the qualitative behavior of uncertain dynamical systems: a computational approach ⋮ Inequalities for the ruin probability in a controlled discrete-time risk process ⋮ On average control generating families for singularly perturbed optimal control problems with long run average optimality criteria ⋮ Central limit theorems for stochastic approximation with controlled Markov chain dynamics ⋮ 𝑊-Markov measures, transfer operators, wavelets and multiresolutions ⋮ Markov control processes with pathwise constraints ⋮ A semidefinite programming approach to the generalized problem of moments ⋮ The heat flow on metric random walk spaces ⋮ The total variation flow in metric random walk spaces ⋮ Rationally Inattentive Control of Markov Processes ⋮ Ergodicity of a bounded Markov chain with attractiveness towards the centre ⋮ A unified stability theory for classical and monotone Markov chains ⋮ Strong Uniform Value in Gambling Houses and Partially Observable Markov Decision Processes ⋮ Robustness of Markov processes on large networks ⋮ A perturbation approach to approximate value iteration for average cost Markov decision processes with Borel spaces and bounded costs ⋮ Stationary and Ergodic Properties of Stochastic NonLinear Systems Controlled over Communication Channels ⋮ Robustness to Incorrect Priors in Partially Observed Stochastic Control ⋮ Unnamed Item ⋮ Convex analytic method revisited: further optimality results and performance of deterministic policies in average cost stochastic control ⋮ Least gradient functions in metric random walk spaces ⋮ Approximations of Countably Infinite Linear Programs over Bounded Measure Spaces ⋮ Unnamed Item ⋮ On the Hill relation and the mean reaction time for metastable processes ⋮ Convergence of Recursive Stochastic Algorithms Using Wasserstein Divergence ⋮ Averaging and linear programming in some singularly perturbed problems of optimal control

This page was built for publication: Markov chains and invariant probabilities