Stochastic approximation of the paths of killed Markov processes conditioned on survival (Q2675985)

It is known that reinforced processes provide a stochastic representation for the quasi-stationary distribution (= QSD) of a killed Markov process. This paper treats the provision of a pathwise description by the construction of reinforced processes and also deals with a stochastic approximation for the quasi-limiting distributins of reducible killed Markov processes. The author considers in discrete time a killed Markov process \((X_t), 0 \leqslant t < \tau_{\partial}\) on the state space \(\chi \cup \partial\), evolving in a metric space \(\chi\) until the killing time \(\tau_{\partial} := \inf \{ t > 0: X_t \in \partial \}\) after time it remains in the cemetery state \(\partial\), where \(\partial\) is assumed to be a one-point set without loss of generality. \par Let \(\mathcal{M}(\chi)\) be the set of Borel measures on \(\chi\), and let \(\mathcal{P}(\chi)\) be the set of Borel probability measures on \(\chi\). Both sets are equipped with the topology of weak convergence of measures. Let \(X_0 \sim \mu \in \mathcal{P}(\chi)\), and \(\mathcal{L}_{\mu}\) denotes the law of \(X_t\). Moreover, let \(C_b(\chi)\) be the space of bounded and continuous functions on \(\chi\). When \(C_0(\chi)\) denotes the space of continuous functions on \(\chi\), vanishing on \(\partial_{\chi} := \bar{\chi} \setminus \chi\), then assume that \[ (\chi \ni x \mapsto K f(x) := K(x, f)) \in C_o(\chi) \] for all \(f \in C_b(\chi)\). Assume also that \(\mathbb{P}_{\mu}(\tau_{\partial} > T) > 0\). Then \(K\) defines a contraction operator \(K : \chi \ni x \mapsto Kf(x) \in C_0(\chi) \subset C_b(\chi)\) with spectral radius \(r(K)\). The parameter \(p\) is chosen reasonably in such a way that \(1 - (1-p)/Z = r(K)\). The reinforced path process \((u_n)_n\) is defined by inductive sampling triples \(u_n = (t_b^n, f^n, t_d^n)\), where \(t_b^n\) is the \(n\)-th birth time, \(t_d^n\) is the \(n\)-th killing time, and \(f^n\) is the \(n\)-th path from time 0 to time \(t_d^n\). Here is the main result of this article. \textbf{Theorem A.} There is a unique solution \(Z\) to \[ z = p \cdot \sum_{s=0}^T \mathbb{P}_{\mu} (\tau_{\partial} > s) \left\{ 1 - \frac{1-p}{z} \right\}^{-(s+1)} \tag{1} \] for \(z \in [1, \infty)\) and \(0 < p \leqslant 1\). Let \(( \gamma_t ), 0 \leqslant t \leqslant T\) (or \((\gamma_t), 0 \leqslant t < \infty\) if \(T = \infty\)), be the coefficient defined by \[ \gamma_t := p \cdot \mathbb{P}_{\mu} (\tau_{\partial} > t) \left\{ 1 - \frac{1-p}{Z} \right\}^{-(T +1)} \tag{2} \] with the solution \(Z\) to (1). Then for any \(t \in [0, T]\) \(( t \in\mathbb{N}\) if \(T = \infty)\) we have \[ \frac{1}{N} \sum_{n=1}^N \mathbf{1}(t_b^n < t < t_d^n) \delta_{ f_{ [0,t]}^n}(\cdot) \quad \xrightarrow{\mathcal{M}( F( [0,t]; \chi))} \quad \gamma_t \cdot \mathcal{L}_{\mu} ( (X_s), 0 \leqslant s \leqslant t \vert \tau_{\partial} > t ) (\cdot), \quad \text{a.s.} \tag{3} \] as \(N \to \infty\). \textit{D. Aldous} et al. introduced in [Probab. Eng. Inf. Sci. 2, No. 3, 293--307 (1988; Zbl 1134.68592)] a method for simulating the quasi-stationary distributions (= QSDs) \(\pi\) based on reinforced processes \((Y_t ), 0 \leqslant t < \infty\), where they established that various kinds of representations of \(Y\) provide an approximation method for the QSDs of irreducible killed Markov chains when the state space is finite and time is discrete. While, \textit{C. Mailler} and \textit{D. Villemonais} [Ann. Appl. Probab. 30, No. 5, 2393--2438 (2020; Zbl 1461.62147)] derived quantitative results for a more general version of the algorithm. However, these results cannot apply to reducible killed Markov processes. Therefore, the author obtains in the following assertion a stochastic approximation for the quasi-limiting distributions of reducible killed Markov processes for given initial condition as a corollary of the above-mentioned main result (= Theorem A). Here is the corollary. \textbf{Corollary B.} Let \(0 < p \leqslant 1\). \(Z^p\) denotes the unique solution to (1). Let \((\gamma_t^p), 0 \leqslant t < \infty\), be the coefficients defined by (2). Then we have \[ \frac{1}{t} \sum_{s=0}^{t-1} \delta_{ Y_s^p} (\cdot) \quad \xrightarrow{\text{a.s.}} \quad \pi^p(\cdot) \] as \(t \to \infty\), whereby \[ \pi^p(\cdot) := \sum_{t=0}^{\infty} \frac{\gamma_t^p}{Z^p} \mathcal{L}_{\mu} (X_t \vert \tau_{\partial} > t)(\cdot) \quad \longrightarrow \quad \pi(\cdot) \] as \(p \to 0\).