Optimal stopping with continuous control of piecewise deterministic Markov processes

From MaRDI portal





Optimization problems related to piecewise-deterministic Markov processes (PDP's) [see \textit{M. H. A.~Davis}, ``Markov models and optimization (1993; Zbl 0780.60002)] are considered. The motion of a PDP depends on three local characteristics: the flow \(\varphi\), the jump rate \(\lambda\) and the transition measure \(Q\), which specifies the post-jump location. Starting from \(x\) the motion of the process follows the flow \(\varphi(t,x)\) until the first jump time \(T_1\) which occurs either randomly in a Poisson-like fashion with rate \(\lambda\varphi(t,x)\) or when the flow hits the boundary of the state space. In both cases the location of the process at the jump time \(T_1\) is selected by the transition measure \(Q(\cdot,\varphi(T_1,x))\). After the jump time the motion restarts from new point. In the optimization problem it is allowed for continuous control on the local parameter \(\lambda\) and \(Q\) and stopping process. For each admissible continuous control \(\psi\in\Psi\), an \({\mathcal F}_t^\psi\)-stopping time \(\tau\) and \(\alpha>0\) the cost \({\mathcal I}^{\psi,\tau}(x)\) is defined. The following optimal stopping problem with continuous control \(\rho(x)=\inf_{\psi\in\Psi}\inf_{\tau\in{\mathcal M}^\psi}{\mathcal I}^{\psi,\tau}(x)\), where \({\mathcal M}^\psi\) denotes the set of all \({\mathcal F}_t^\psi\)-stopping times, is considered. Optimality equations are obtained in terms of a set of variational inequalities as well as on the first jump time operator of the PDP. It is shown that if the final cost function is absolutely continuous along trajectories, then the value function of the optimal stopping problem with continuous control is also absolutely continuous along trajectories. The optimal stopping problems for PDP have been considered by \textit{O. L. V. Costa} and \textit{M. H. A. Davis} [Math. Control Signals Syst. 1, No. 2, 123-146 (1988; Zbl 0657.93077)], \textit{D. Gątarek} [Appl. Math. Optimization 24, No. 1, 85-98 (1991; Zbl 0746.49013)] and \textit{U. S. Gugerli} [Stochastics 19, 221-236 (1986; Zbl 0611.60039)]. The continuous control models were investigated, among others, by \textit{M. A. H. Dempster} [in: Applied stochastic analysis. Stochastic Monogr. 5, 303-325 (1990; Zbl 0756.90092)] and \textit{D. Vermes} [Stochastics 14, 165-207 (1985; Zbl 0566.93074)]. The results of the paper unify and generalize previous ones known in the mentioned literature.











This page was built for publication: Optimal stopping with continuous control of piecewise deterministic Markov processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4504922)