A formula for the superdifferential of the distance determined by the gauge function to the complement of a convex set (Q2304483)

Let \(k(\cdot)\) be the Minkowski gauge functional of a convex compact subset \(M\) of \(\mathbb{R}^p\) with \(0\in \mathrm{int}\,M\). Then \(k(\cdot)\) is an asymmetric norm on \(\mathbb{R}^p \) (see [\textit{S. Cobzaş}, Functional analysis in asymmetric normed spaces. Basel: Birkhäuser (2013; Zbl 1266.46001)]). For \(\Omega\subset \mathbb{R}^p\), the distance from a point \(x\in\mathbb{R}^p\) to \(\Omega\) is given by \(\rho(x,\Omega)=\inf\{k(x-y):y\in \Omega\}\). The author shows that the distance from \(x\) to a closed semispace \(\Omega=\{y\in\mathbb{R}^p:\langle a,y-b\rangle \le 0\}\) is given by \(\rho(x,\Omega)=\langle a, x-b\rangle/s(x,M)\), where \(s(x,M)=\sup\{\langle z,x\rangle: z\in M\}\). Let \(\Omega=\overline{\mathbb{R}^p\setminus D}\), where \(D\subset \mathbb{R}^p\) is closed convex with \(\mathrm{int}\,D\ne\emptyset\). The main result of the paper is Theorem~2, which shows that the distance function to \(\Omega \) is concave on \(D\) and gives formulae for the directed derivatives and for the superdifferential of the distance function \(\rho(\cdot,\Omega)\) with respect to the asymmetric norm \(k(\cdot)\), expressed in terms of the cone of feasible directions to the set \(D\) and of the support function \(s(x,M)\). Recall that the directional derivative of a function \(f:\mathbb{R}^p\to \mathbb{R}\) at \(x\) in the direction \(g\in\mathbb{R}^p\) is given by \(f'(x;g)=\lim_{\alpha\searrow 0}\alpha^{-1}\big(f(x+\alpha g)-f(x)\big).\) One says that the function \(f\) is superdifferentiable at \(x\in\mathbb{R}^p\) if the directional derivatives exist for all \(g\in\mathbb{R}^p\) and the function \(f'(x;\cdot)\) is superlinear (i.e., positively homogeneous and superadditive). In this case, there exists a unique compact convex subset \(\bar \partial f(x)\subset\mathbb{R}^p\) (called the superdifferential of \(f\) at~\(x\)) such that \( f'(x;g)=\min\{\langle x,v\rangle: v\in \bar\partial f(x)\}\) (see, for instance, [\textit{V. F. Dem'yanov} and \textit{A. M. Rubinov}, Quasidifferential calculus. New York: Optimization Software, Inc., Publications Div. (1986; Zbl 0712.49012)]).