Optimality conditions in mathematical programming and composite optimization (Q1341569)

In a very clear manner and with all important proofs new second-order necessary optimality conditions are derived for the problems \(({\mathcal M})\): \(j(x)\to \inf\), s.t. \(x\in B\cap k^{- 1}(C)\), and \(({\mathcal C})\): \(f(x):= g(h(x))\to \inf\), s.t. \(x\in D\) where \(j: X\to \mathbb{R}\), \(k: X\to Z\) are twice differentiable at some point \(a\) of \(F:= B\cap k^{- 1}(C)\), \(B\) and \(C\) being closed convex subsets of the Banach spaces \(X\), and \(Z\), resp. and where \(h: X\to Y\) is twice differentiable, \(g: Y\to \mathbb{R}\) is a closed proper convex function, \(Y\) is a Banach space and \(D\) is a closed convex subset of \(X\). At first the author derives a general second-order optimality condition for the problem \(({\mathcal P})\): \(f(x)\to \inf\), s.t. \(x\in F\) where \(f: X\to \mathbb{R}\) defines a continuous linear map \(f'(a): X\to \mathbb{R}\) and a continuous bilinear map \(f''(a): X\times X\to \mathbb{R}\) such that \(\lim_{(t, u)\to (0, x)} t^{- 2}[f(a+ tu)- f(a)- tf'(a) u- 0.5t^2 f''(a)uu]= 0\) for all \(x\in X\) and where the feasible set \(F\) belongs to an arbitrary topological vector space \(X\). Using the contingent cone \(F'(a)\) to \(F\) at \(a\in F\) and the (superior) second-order tangent set \(F''(a, v)\) to \(F\) at \(a\in F\) in the direction \(v\in X\) defined by \(F''(a, v):= \limsup_{t\to 0+} 2t^{- 2}(F- a- tv)\) the following second-order condition holds: If \(a\) is a local solution of \(({\mathcal P})\) then (i) \(f'(a)v\geq 0\) for each \(u\in F'(a)\) and (ii) \((a)vv+ \lim_{(t, u)\to (0, x), a+ tu\in F} f'(a)t^{- 1}(u- v)\geq 0\) for each \(v\in F'(a)\cap \ker f'(a)\). Under metrical regularity for the mapping \(k\) (some Ljusternik-like condition) and some Zowe/Kurcyusz-like regularity condition of \(k'\) w.r.t. \(B'(a)\) and \(C'(k'(a))\) a second-order condition of Lagrange type is derived from the above geometrical condition for \(({\mathcal M})\). Further the equivalence of the problems \(({\mathcal M})\) and \(({\mathcal C})\) is shown. After considering compound tangent sets and compound derivatives of second-order, a second-order necessary condition for \(({\mathcal C})\) is again derived from the above geometrical condition. If the space \(X\) is finite-dimensional then the strict inequality in (ii) (and in the other not mentioned conditions of second-order) gives second- order sufficient conditions of optimality.