Stochastic approximation of the paths of killed Markov processes conditioned on survival
It is known that reinforced processes provide a stochastic representation for the quasi-stationary distribution (= QSD) of a killed Markov process. This paper treats the provision of a pathwise description by the construction of reinforced processes and also deals with a stochastic approximation for the quasi-limiting distributins of reducible killed Markov processes. The author considers in discrete time a killed Markov process \((X_t), 0 \leqslant t < \tau_{\partial}\) on the state space \(\chi \cup \partial\), evolving in a metric space \(\chi\) until the killing time \(\tau_{\partial} := \inf \{ t > 0: X_t \in \partial \}\) after time it remains in the cemetery state \(\partial\), where \(\partial\) is assumed to be a one-point set without loss of generality. \par Let \(\mathcal{M}(\chi)\) be the set of Borel measures on \(\chi\), and let \(\mathcal{P}(\chi)\) be the set of Borel probability measures on \(\chi\). Both sets are equipped with the topology of weak convergence of measures. Let \(X_0 \sim \mu \in \mathcal{P}(\chi)\), and \(\mathcal{L}_{\mu}\) denotes the law of \(X_t\). Moreover, let \(C_b(\chi)\) be the space of bounded and continuous functions on \(\chi\). When \(C_0(\chi)\) denotes the space of continuous functions on \(\chi\), vanishing on \(\partial_{\chi} := \bar{\chi} \setminus \chi\), then assume that \[ (\chi \ni x \mapsto K f(x) := K(x, f)) \in C_o(\chi) \] for all \(f \in C_b(\chi)\). Assume also that \(\mathbb{P}_{\mu}(\tau_{\partial} > T) > 0\). Then \(K\) defines a contraction operator \(K : \chi \ni x \mapsto Kf(x) \in C_0(\chi) \subset C_b(\chi)\) with spectral radius \(r(K)\). The parameter \(p\) is chosen reasonably in such a way that \(1 - (1-p)/Z = r(K)\). The reinforced path process \((u_n)_n\) is defined by inductive sampling triples \(u_n = (t_b^n, f^n, t_d^n)\), where \(t_b^n\) is the \(n\)-th birth time, \(t_d^n\) is the \(n\)-th killing time, and \(f^n\) is the \(n\)-th path from time 0 to time \(t_d^n\). Here is the main result of this article. \textbf{Theorem A.} There is a unique solution \(Z\) to \[ z = p \cdot \sum_{s=0}^T \mathbb{P}_{\mu} (\tau_{\partial} > s) \left\{ 1 - \frac{1-p}{z} \right\}^{-(s+1)} \tag{1} \] for \(z \in [1, \infty)\) and \(0 < p \leqslant 1\). Let \(( \gamma_t ), 0 \leqslant t \leqslant T\) (or \((\gamma_t), 0 \leqslant t < \infty\) if \(T = \infty\)), be the coefficient defined by \[ \gamma_t := p \cdot \mathbb{P}_{\mu} (\tau_{\partial} > t) \left\{ 1 - \frac{1-p}{Z} \right\}^{-(T +1)} \tag{2} \] with the solution \(Z\) to (1). Then for any \(t \in [0, T]\) \(( t \in\mathbb{N}\) if \(T = \infty)\) we have \[ \frac{1}{N} \sum_{n=1}^N \mathbf{1}(t_b^n < t < t_d^n) \delta_{ f_{ [0,t]}^n}(\cdot) \quad \xrightarrow{\mathcal{M}( F( [0,t]; \chi))} \quad \gamma_t \cdot \mathcal{L}_{\mu} ( (X_s), 0 \leqslant s \leqslant t \vert \tau_{\partial} > t ) (\cdot), \quad \text{a.s.} \tag{3} \] as \(N \to \infty\). \textit{D. Aldous} et al. introduced in [Probab. Eng. Inf. Sci. 2, No. 3, 293--307 (1988; Zbl 1134.68592)] a method for simulating the quasi-stationary distributions (= QSDs) \(\pi\) based on reinforced processes \((Y_t ), 0 \leqslant t < \infty\), where they established that various kinds of representations of \(Y\) provide an approximation method for the QSDs of irreducible killed Markov chains when the state space is finite and time is discrete. While, \textit{C. Mailler} and \textit{D. Villemonais} [Ann. Appl. Probab. 30, No. 5, 2393--2438 (2020; Zbl 1461.62147)] derived quantitative results for a more general version of the algorithm. However, these results cannot apply to reducible killed Markov processes. Therefore, the author obtains in the following assertion a stochastic approximation for the quasi-limiting distributions of reducible killed Markov processes for given initial condition as a corollary of the above-mentioned main result (= Theorem A). Here is the corollary. \textbf{Corollary B.} Let \(0 < p \leqslant 1\). \(Z^p\) denotes the unique solution to (1). Let \((\gamma_t^p), 0 \leqslant t < \infty\), be the coefficients defined by (2). Then we have \[ \frac{1}{t} \sum_{s=0}^{t-1} \delta_{ Y_s^p} (\cdot) \quad \xrightarrow{\text{a.s.}} \quad \pi^p(\cdot) \] as \(t \to \infty\), whereby \[ \pi^p(\cdot) := \sum_{t=0}^{\infty} \frac{\gamma_t^p}{Z^p} \mathcal{L}_{\mu} (X_t \vert \tau_{\partial} > t)(\cdot) \quad \longrightarrow \quad \pi(\cdot) \] as \(p \to 0\).
- General approximation method for the distribution of Markov processes conditioned not to be killed
- Stochastic approximation of quasi-stationary distributions for diffusion processes in a bounded domain
- Quasi-stationary distributions: an application of the revival technique
- Quasi-stationary distributions for birth-death processes with killing
- An approximation scheme for quasi-stationary distributions of killed diffusions
- A Fleming-Viot particle representation of the Dirichlet Laplacian
- A stochastic approximation approach to quasi-stationary distributions on finite spaces
- Exponential convergence to quasi-stationary distribution and \(Q\)-process
- General approximation method for the distribution of Markov processes conditioned not to be killed
- Stochastic approximation of quasi-stationary distributions for diffusion processes in a bounded domain
- Stochastic approximation on noncompact measure spaces and application to measure-valued Pólya processes
- The distribution of the spine of a Fleming-Viot type process
- Two Applications of Urn Processes The Fringe Analysis of Search Trees and The Simulation of Quasi-Stationary Distributions of Markov Chains
- Uniform convergence to the \(Q\)-process
- Likelihood ratio processes for Markovian particle systems with killing and jumps
- Stochastic approximation of quasi-stationary distributions for diffusion processes in a bounded domain
- An approximation scheme for quasi-stationary distributions of killed diffusions
- Killed Markov Decision Processes on Finite Time Interval for Countable Models
- scientific article; zbMATH DE number 3917393 (Why is no real title available?)
- Survival probability of mutually killing Brownian motions and the O'Connell process
- Approximating quasi-stationary distributions with interacting reinforced random walks
- General approximation method for the distribution of Markov processes conditioned not to be killed
- Recent advances in the long-time analysis of killed degenerate processes and their particle approximation
- Stochastic Approximation of the Paths of Killed Markov Processes Conditioned on Survival
This page was built for publication: Stochastic approximation of the paths of killed Markov processes conditioned on survival
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2675985)
