Numerical methods for the exit time of a piecewise-deterministic Markov process

DOI10.1239/AAP/1331216650MaRDI QIDQ2879913zbMATH OpenOpenAlexFDO

Authors Adrien Brandejsky, Benoîte de Saporta, F. Dufour

Publication date 10 April 2012

Published in Advances in Applied Probability (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1012.2659

exit time numerical method piecewise deterministic Markov process quantization

Computational methods in Markov chains (60J22) Continuous-time Markov processes on general state spaces (60J25) Applications of renewal theory (reliability, demand theory, etc.) (60K10) Probabilistic models, generic numerical methods in probability and statistics (65C20)

Abstract: We present a numerical method to compute the survival function and the moments of the exit time for a piecewise-deterministic Markov process (PDMP). Our approach is based on the quantization of an underlying discrete-time Markov chain related to the PDMP. The approximation we propose is easily computable and is even flexible with respect to the exit time we consider. We prove the convergence of the algorithm and obtain bounds for the rate of convergence in the case of the moments. An academic example and a model from the reliability field illustrate the paper.

Review text

The paper introduces a method for numerically approximating the distribution and the moments of the first exit time of a piecewise deterministic process (PDMP) from a subset of its state space. The authors present direct approximation methods for both problems, although the moment approximations can in principle be calculated from the approximation to the distribution. In general, the method is based on the quantization of the embedded Markov chain of jump times and post-jump locations. The presented approach is flexible with respect to the domain for which the exit time is calculated in as such as the time-consuming initial problem of calculating the quantization depends only on the process and not on the domain. Storing these results off-line, the distribution and moments of exit times for different domains can be approximated very quickly. This provides an advantage over Monte-Carlo methods.NEWLINENEWLINEIn brief the method is described as follows. First, the authors approximate the exit time \(\tau\) by \(\tau\wedge \tau_N\), where \(\tau_N\) is the \(N\)th jump time of the PDMP -- for large enough \(N\), as \(\tau_N\to\infty\) almost surely is assumed, this is expected to be a reasonable approximation. The question of choosing the time horizon \(\tau_N\) is only marginally addressed in the study. Next, the problem of calculating the survivor function of \(\tau\wedge\tau_N\) and its moment is reduced to calculate two sequences which can be obtained from the Markov chain. Then the quantized Markov chain is substituted, resulting in approximations to these sequences which then in turn provide approximation to the desired quantities. The study contains a detailed list of assumptions on the local characteristics defining PDMPs and quantities related to the exit time problems under which the authors prove convergence of the quantized sequences to the exact sequences when the quantization error converges to zero. This then extends to the pointwise convergence almost everywhere of the survivor function of the exit time and to the convergence of the moments. For the latter, additionally, also a rate of convergence is obtained.NEWLINENEWLINEThe findings are illustrated by two numerical examples. The first, a simple Poisson process example allows for the exact calculation of the distribution of the exit time. The authors compare the exact and approximate survivor functions and also the first and second moments. They find good agreement with their theoretical results. The second, more realistic example is a corrosion model. The authors show that the conditions of their theorems are satisfied. They calculate approximations of the distribution and the mean exit time and compare them to Monte-Carlo estimates of these quantities. Again a good agreement with the theory is found.

Recommendations

Cites work

Cited in

(9)

This page was built for publication: Numerical methods for the exit time of a piecewise-deterministic Markov process

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2879913)