Pitfalls in applying optimal control to dynamical systems: an overview and editorial perspective (Q2081952)

The dynamical systems are \[ x'(t) = f(x(t), u(t)) \, , \quad x(0) = x_0 \, , \quad 0 \le t \le T \] where \(x(t) \in M\) (a manifold in \(\mathbb{R}^n)\) and \(u(t) \in U \subseteq\mathbb{R}^m;\) the time interval \(0 \le t \le T\) may be fixed or variable. The target condition is \((T, x(T)) \in N \subseteq [0, \infty) \times M\) defined by \(\Psi(t, x) = 0\) \((\Psi\) a vector valued function). The objective is to minimize \[ J(u) = \int_0^T L(x(s), u(s)) ds + \varphi(T, x(T)) \] in the space \(\mathcal{U}\) of controls \(u(\cdot)\) that take values in \(U\). Pontryagin's minimum principle gives a necessary condition for a minimum \(\bar u(t)\) using the Hamiltonian \(H(\lambda, x, u) = L(x, u) + \langle \lambda, f(x, u)\rangle;\) there exists a covector \(\lambda(t)\) satisfying the adjoint equation \[ \lambda'(t) = - \frac{\partial H}{\partial x}(\lambda(t), \bar x(t), \bar u(t)) \] (\(\bar x(t)\) the trajectory corresponding to \( \bar u(t))\) such that \((H, - \lambda + \partial \varphi / \partial x)\) is orthogonal to \(N\) at the terminal point and the optimal control satisfies \[ H(\lambda(t), \bar x(t), \bar u(t)) = \min_{u \in U} H(\lambda(t), \bar x(t), u). \tag{1} \] This reduces the infinite dimensional control problem to finite dimension, although not explicitly since the function to be minimized depends on \(\bar u(t)\), \(\bar x(t)\). However, in certain situations (1) can be manipulated into giving an actual solution. It can also be used as a basis for iteration methods. The subject of this expository paper is the case where (1) does not provide enough information to identify the optimal control (for instance, the min is not unique, or there are subsets of \(0 \le t \le T\) where (1) is empty). This may lead to extremals which are not true minima. The authors also discuss the consequences of certain choices of cost functionals (i.e. quadratic vs. affine in the controls \(u_j)\) and the relation of the results with the Hamilton-Jacobi-Bellman approach. There are also observations on numerical approximation.