Minimizing curves in prox-regular subsets of Riemannian manifolds (Q2670985)

Prox-regular sets were introduced by \textit{R. A. Poliquin} and \textit{R. T. Rockafellar} [Trans. Am. Math. Soc. 348, No. 5, 1805--1838 (1996; Zbl 0861.49015)]. The authors of the paper under review have presented in a previous paper [\textit{M. R. Pouryayevali} and \textit{H. Radmanesh}, J. Convex Anal. 26, No. 2, 617--633 (2019; Zbl 1420.58004)] a characterization, in finite dimensional Riemannian manifolds, of prox-regular sets in terms of \(\varphi\)-convex sets, by showing that the two notions coincide in this setting. The notion of \(\varphi\)-convexity was introduced by \textit{M. Degiovanni} et al. [Ric. Mat. 32, 285--319 (1983; Zbl 0555.49007)]. In this paper minimizing curves in prox-regular subsets of Riemannian manifolds are studied. To this purpose the authors investigate the directional differentiability of the metric projection. They initially show, by applying Shapiro variational principle, see e.g. [\textit{A. Shapiro}, SIAM J. Optim. 4, No. 1, 130--141 (1994; Zbl 0793.41021)], that the projection map of a prox-regular subset of a Riemannian manifold is locally Lipschitz on an open neighborhood, generalizing a result of \textit{S. Hosseini} and \textit{M. R. Pouryayevali} [Proc. Am. Math. Soc. 141, No. 1, 233--244 (2013; Zbl 1277.58005)]. Further, they prove that the metric projection is directionally differentiable at the boundary points of prox-regular sets, and give a characterization of the Bouligand tangent cone to a prox-regular set, for the involved definition, recalled in the paper, see e.g. the above quoted paper of Hosseini, S. and Pouryayevali, M. R. In the final part of the paper the authors provide a necessary condition for a curve to be minimizing in a prox-regular set whose boundary is a twice-continuously differentiable submanifold of the given Riemannian manifold. The paper concludes by handling the specific cases of the 2-sphere and the hyperbolic plane.