Optimal control of infinite-dimensional piecewise deterministic Markov processes: a BSDE approach. Application to the control of an excitable cell membrane (Q2232766)

Piecewise deterministic Markov processes (PDMPs) are considered. PDMP is a continuous-time controlled process on the horizon \(]0,T]\) and the state space \(E\), which is a separable Hilbert space. The space of control actions \(A\) is a compact Polish space. Let \((t_n,e_n)_{n\geq 1}\) be the sequence of jumps which occur at the time instants \(t_1 < t_2 \dots <t_n< \dots\) to the states \(e_1,e_2,\dots,e_n,\dots\) where all \(e_i \in E\). Given starting point \((t,x) \in [0,T]\times E\), the set of admissible control laws has the form \[\alpha_s=\alpha_0(s-t,x)\mathbb{I}_{(t,t_1]}(s)+\sum_{n=1}^\infty \alpha_n(s-t_n,e_n)\mathbb{I}_{(t_n,t_{n+1}]}(s),\] where \(t_1>t\), \(\alpha_n:\mathbb{R}_+\times E \to A\), \(\mathbb{I}_{(t_n,t_{n+1}]}(s)=1\) if \(s \in (t_n,t_{n+1}]\) and \(\mathbb{I}_{(t_n,t_{n+1}]}(s)=0\) otherwise. A controlled process is \(X_s=\phi^{\alpha_0}(s-t,x)\) if \(s \in[t,t_1)\), \(X_s=\phi^{\alpha_n}(s-t_n,e_n)\) if \(s \in[t_n,t_{n+1})\), \(n \in\mathbb{N}\), and \(X_s=\Delta\) if \(s >t_\infty=\lim_{n \to \infty} t_n\), where \(\Delta \notin E\) is an isolated point adjoint to \(E\). Here \(\phi^\alpha(s,x)\) is the unique mild solution to the parabolic partial differential equation \(\dot{x}(s)=-L x(s)+b(x(s),\alpha(s))\), \(x(0)=x\), where \(L\) is a linear operator on \(E\) and \(b(x,a)\) is a bounded continuous function. On the process, the probability measure \(\mathbb{P}^{t,x}_\alpha\) is defined. The cost functional is \[J(t,x,\alpha)=\mathbb{E}^{t,x}_\alpha \left[\int_t^T f(X_s,\alpha_s) ds+g(X_T)\right], \]where \(\mathbb{E}^{t,x}_\alpha\) means mathematical expectation with respect to \(\mathbb{P}^{t,x}_\alpha\). The value function is \(V(t,x)=\inf_\alpha J(t,x,\alpha)\).\par In the article, it is proved that the value function can be represented through a Feynman-Kac formula. This representation is obtained using implementing the randomization procedure. The value function of this randomized control problem can be represented by means of the solution of a constrained backward stochastic differential equation. An important intermediate step is to show that the corresponding Hamilton-Jacobi-Bellmann equation has a unique viscosity solution by providing a suitable comparison theorem. The results are applied to the control of a PDMP Hodgkin-Huxley model with spatial component.