A nonconvex separation property in Banach spaces (Q1298712): Difference between revisions

The well-known separation theorem points out that the convex sets \(Z_1,Z_2\in X\) (\(X\) a Banach space) with \(0\not\in \text{int}(Z_1- Z_2)\) are separable by a closed hyperplane, i.e., there exists a function \(p^*\in X^*\setminus\{0\}\) with \[ \langle p^*, z_1\rangle\leq \langle p^*, z_2\rangle\quad\text{for all }z_1\in Z_1\;\text{and} z_2\in Z_2. \] Clearly, for \(z\in Z_1\cap Z_2\neq \emptyset\) we have \[ p^*\in N_{Z_1}(z)= \text{cl cone }\partial d_{Z_1}(z),\quad -p^*\in N_{Z_2}(z)= \text{cl cone }\partial d_{Z_2}(z) \] using the classical normal cones, the distance functions and their subdifferentials associated to the sets \(Z_1\) and \(Z_2\), respectively. In the present paper, this result is extended for nonconvex closed sets introducing an abstract definition of subdifferential. The main theorem, formulated as a limiting result and proved by means of Ekeland's variational principle, can be simplified if one of the sets is assumed to be (locally) compactly epi-Lipschitz. Moreover, it is shown that the assertion is closely connected with the extremal principle of Mordukhovich. At the end of the paper the authors give some consequences in optimization regarding the existence of singular multipliers.