Stochastic control for a class of random evolution models (Q1879225)

The following stochastic control problem is considered. Let \(X(s)\) describe the random evolution given by \[ \frac{d}{ds} X(s)= I(i(s)), \quad X(t)=x \in R, \quad t \leq s \leq \tau, \] where \(i(s) \in \{-1,+1\}\) is a Markovian alternating renewal process with \(I(-1)=a\) and \(I(+1)=b\). The holding times in states \(a\) and \(b\) are respectively given by two control variables \(u_{-1}(s)\) and \(u_{+1}(s)\), so as the couple \(\tilde{u}(s)=(u_{-1}(s),u_{+1}(s))\) can be considered as the control variable. The goal is to minimize by \([\tilde{u}(s); t\leq s\leq \tau]\) the functional \[ J(x,t,i,\tilde{u})=E\left\{\int_t^\tau L((i(s),\tilde{u}(s)) ds +\psi (i(\tau),X(\tau)) \;| \;i(t)=i, X(t)=x \right\}, \] where \(\psi(i(\tau),X(\tau))\) stands for the final cost at time \(\tau\) and \( L(i,u)=u_i \ln u_i -u_i+1\), \(i \in \{-1,+1\}. \) The associated dynamic programming equations are drawn and the optimal control functions \(\tilde{u}(s)=(u_{-1}(s),u_{+1}(s))\) are represented. Moreover, it is shown that these equations are equivalent to a completely solvable model of the nonlinear Boltzmann equation constructed by Ruijgrok and Wo. A short review of related results is given.