A generalized Hamilton-Jacobi-Bellman equation for deterministic optimal control problems

Publication date 15 September 1999

direction vector generalized Hamilton-Jacobi-Bellman equation lower tangential derivative optimal control value function viscosity solution

Mathematics Subject Classification ID

Optimality conditions for problems involving ordinary differential equations (49K15) Dynamic programming in optimal control and differential games (49L20) Viscosity solutions to Hamilton-Jacobi equations in optimal control and differential games (49L25)

Review text

The author proves several theorems concerning, mainly, the value function \(W(.,.)\) of a Mayer-type optimal control problem which consists in minimizing \(g(x(1))\) subject to: \[ x'(t)=f(t,x(t),u(t)) \text{ a.e. }([\tau ,1]), \;x(\tau)=\xi, \;\tau <1, \;x(1)\in {\mathcal T}_1 \subset R^n, \;u(t)\in Z \subset R^n \tag{1} \] for every initial point \((\tau,\xi)\in {\mathcal D}\subset [0,1]\times R^n\). The results in the paper are expressed in terms of two new concepts introduced by the author: if \({\mathcal R}(t;\tau ;\xi):=\{x(t)\); \(x(.)\) a solution of (1)\}, \({\mathcal R }(\tau,\xi):= \cup_{t\in [\tau,1]}{\mathcal R} (t;\tau,\xi)\) denote the rechable sets of \((1)\) then \(v\in R^n\) is said to be a direction vector to \({\mathcal R}(\tau,\xi)\) at \((\tau,\xi))\) if there exists a ``secant sequence, \(\sigma :=\{(s_k,x_k)\}\in \Sigma (v)\), \((s_k,x_k)\to (o_+,\xi) \) such that: \[ x_k \in {\mathcal R}(\tau +s_k;\tau ,\xi), \quad{{x_k -\xi}\over{s_k}}\to v. \tag{2} \] On the other hand if \(v\in R^n\) is a direction vector to \({\mathcal R}(\tau,\xi)\) at \((\tau,\xi)\) then the lower tangential derivative of the real function \(L(.,.): {\mathcal D}\to R\) at \((\tau,\xi)\) in direction \((1,v)\) is defined by: \[ D_T^-(L(\tau,\xi);(1,v)):=\inf \Biggl\{ \liminf_{k\to \infty} {{L(\tau +s_k,x_k)-L(\tau,\xi)}\over{s_k}}; \;\sigma =\{(s_k,x_k)\} \in \Sigma (v)\Biggr\}.\tag{3} \] Under certain rather restrictive hypotheses the author proves in Theorem 2.1 that the set of direction vectors to \({\mathcal R}(\tau,\xi)\) at \((\tau,\xi)\) coincides with \(F(\tau ,\xi):=f(\tau ,\xi ,Z)\) and in Theorem 3.2 the fact that the restriction to \(\text{Int}({\mathcal D})\) of the value function \(W(.,.)\) is a viscosity solution of the associated Hamilton-Jacobi-Bellman equation: \[ -W_t(t,x)+H(t,x,-W_x(t,x))=0, \;(t,x)\in {\mathcal D}, \;W(1,x)=g(x), \;H(t,x,p):=\sup_{z\in Z} \langle p,f(t,x,z)\rangle.\tag{4} \] As main result of the paper may be considered the one contained in Theorems 3.1 and 3.3 according to which a continuous function \(V(.,.):{\mathcal T} \cup \text{Int}({\mathcal D})\to R\) coincides with the restriction to this set of the value function if and only if it satisfies: \[ \min_{v\in F(\tau,\xi)}D_T^-(V(\tau ,\xi);(1,v))=0, \;(\tau ,\xi)\in \text{Int}({\mathcal D}), \;V(1,\xi)=g(\xi), \;\xi \in {\mathcal T}_1. \tag{5} \] One may note, however, that the set of ``direction vectors in (2) has been introduced in [\textit{St. Mirica}, Rev. Roum. Math. Pures Appl. 37, No. 5, 421-444 (1992; Zbl 0766.34011)] as the set \(GK_F^1(\tau ,\xi)\) of generalized 1-contingent directions to trajectories of the differential inclusion \(x'\in F(t,x)\) where Lemma 5.1 contains a stronger result than author's Theorem 2.1; a similar set of (unilateral) tangent directions has been introduced also by \textit{E. Roxin} [J. Differ. Equations 1, 188-205 (1965; Zbl 0135.30802)]. On the other hand, in Sections 1,2 the author suggests that ``there is no loss of generality in replacing a general ``terminal set \({\mathcal T} \subset R \times R^n\) by one of the form \({\mathcal T} =\{1\}\times {\mathcal T}_1, \;{\mathcal T}_1 \subset R^n\) (defining a problem with fixed ``terminal time); however, author's result in Theorem 3.3 may not be true for variable terminal time problems as Example 9 in [\textit{St. Mirica}, An. Ştiinţ. Univ. Al. I. Cuza Iaşi, Mat. 38, No. 1, 89-102 (1992; Zbl 0823.49020)] shows; if \(f(t,x,u)\equiv u\in F(t,x)=Z:= \text{co}\{ (v,v^2); \;v\in [-2,2]\}, \;{\mathcal T}:= \{(\tau ,(\tau ,\xi_2)); \;\tau \in R, \;\xi_2 \geq 0\}, \;g(\tau ,(\tau ,\xi_2)):=\xi_2, \;{\mathcal D}:=\{(t,(x_1,x_2)); \;x_1\leq t, \;x_2 \geq 4(x_1-t)\}\) then hypotheses A.1-A.5 are satisfied, the value function is given by: \(W(t,x):= x_2+4(t-x_1)\) but the function \(V(t,x):=x_2\) satisfies also (5) as well as the corresponding Hamilton-Jacobi-Bellman equation in (4). For the entire collection see [Zbl 0905.00042].

Recommendations

Cited in

(15)

This page was built for publication: A generalized Hamilton-Jacobi-Bellman equation for deterministic optimal control problems

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4227205)