Subgradients of distance functions at out-of-set points (Q2495225)

The authors provide import (generalized) differentiability properties for the classical distance function \[ d(x;\Omega):=\text{inf}\{\| y- x\|\mid y\in\Omega\} \] to a closed set \(\Omega\subset X\) and especially for the general distance function \[ \rho(x; z):= d(x; F(z))= \text{inf}\{\| y-x\|\mid y\in F(z)\} \] to a set-valued mapping \(F: Z\rightrightarrows X\). All spaces are assumed to be Banach spaces. In the first part, upper estimates for various subdifferentials (the \(\varepsilon\)-subdifferential, the Fréchet subdifferential, the proximal subdifferential, the limiting subdifferential and the Dini-Hadamard subdifferential) to these functions at out-of-set points are presented. The results are extended in the second part with the introduction of a new type of right-sided limiting subgradients, and in the third part with the discussion of singular subgradients of marginal functions and general distance functions and their representations by means of the mixed coderivatives of the associated set-valued mappings. Finally in the last part, two applications of the main results are pointed out regarding sufficient conditions for the projection nonemptiness of set-valued mappings and for Lipschitzian continuity of the general distance function.