The maximum principle for discrete-time control systems and applications to dynamic games (Q2633734)

In finite dimension, the weak maximum principle (system of adjoint equations, stationarity of the Hamilton w.r.t. the controls $u_k$, limiting transversality condition) is derived for a discrete time control problem with infinite horizon and without state constraints whenever the optimal control $\hat u_k$ belongs to the interior of the control set $U_k$ for $k=0,1,2,\dots$. Specialization to finite horizon is given. Using the chain rule and uniform convergence of constructed infinite series the costates are given explicitly in Lemma 5. If the control sets are convex and the objective reduced by the state equations is concave (hard to verify) then the necessary conditions are sufficient too. The principles are applied to a consumption-investment problem and a linear regulator problem. An extension to state dependent controls is done. The extension is applied to optimal economic growth. Finally without proof, the extension to dynamic games is similarly executed and a game with linear dynamic illustrates the results. In both extensions, the optimal controls belong to the interior of the related control sets too.