The Bregman distance, approximate compactness and convexity of Chebyshev sets in Banach spaces (Q979032)

Let \(X\) be a real Banach space and \(g:X\to\bar{\mathbb R}=\mathbb R\cup\{\infty\}\) be a proper convex function. The Bregman distance corresponding to \(g\) is defined by \(D_g(y,x)=g(y-g(x)-g'_+(x;y-x)\) for \(x,y\in\text{dom}\,g\), where \(g'_+(x;\cdot)\) stands for the right-hand directional derivative of \(g\) at \(x\). The paper is concerned with the famous problem of convexity of Chebyshev sets -- a subset \(C\subset \) int (dom \(g\)) is called \(D_g\)-Chebyshev if the \(D_g\)-projection \(\Pi^g_C(x)= \) arg min\(_{y\in C} D_g(y,x)\) is single-valued for every \(x\in\) dom \(g\). It is still unknown whether a Chebyshev subset of a Hilbert space \(X\) is convex or not. If \(X\) is finite-dimensional, then the answer is yes, a result that also holds in finite-dimensional smooth normed spaces. Some positive results, strongly connected with the geometric properties of the underlying Banach space, are known in infinite dimensions, too, see the survey by \textit{V.\,S.\thinspace Balaganskij} and \textit{L.\,P.\thinspace Vlasov} [Russ.\ Math.\ Surv.\ 51, No.\,6, 1127--1190 (1996; Zbl 0931.41017), translation from Usp.\ Mat.\ Nauk 51, No.\,6, 125--188 (1996); errata ibid.\ 52, No.\,1, 237 (1997)]. These results were extended by \textit{H.\,Bauschke, X.\,Wang, J.\,Ye} and \textit{X.\,Yuan} [J.~Approx.\ Theory 159, No.\,1, 3--25 (2009; Zbl 1175.49017)] to \(D_g\)-Chebyshev sets in the \(n\)-dimensional Euclidean space, supposing that the function \(g\) satisfies some adequate conditions. The aim of the present paper is to further extend some of these results to infinite-dimensional reflexive Banach spaces, giving, at the same time, generalizations of the corresponding results on the convexity of Chebyshev sets in Banach spaces as well as of some results of \textit{Y.\,I.\thinspace Alber} [Lect.\ Notes Pure Appl.\ Math.\ 178, 15--50 (1996; Zbl 0883.47083)] on generalized projection operators. A key role in this study is played by the notions of \(D_g\)-approximative compactness and of weak \(D_g\)-approximative compactness. The differentiability properties of the function \(g\) have a strong impact on the continuity of the Bregman projection \(\Pi^g_C\) and of its conjugate \(P^g_C(x^*)=\) arg min\(_{y\in C} W^g(y,x^*)\), where \(W^g(y,x^*)=g(x)-\langle x^*,x\rangle + g^*(x^*)\), with \(g^*\) being the Fenchel conjugate of \(g\). In their turn, as in the case of Chebyshev sets in Banach spaces, these continuity properties are tightly connected with the convexity properties of the \(D_g\)-Chebyshev sets. For instance, let \(X\) is a reflexive Banach space, \(g\) a coercive (i.e., \(\lim_{\|y\|\to\infty}g(y)/\|y\|=1\)) essentially smooth function and \(C\subset \) int (dom \(g\)) a \(D_g\)-Chebyshev set. Then the following assertions are equivalent: (i) The set \(C\) is convex; (ii) the set \(C\) is weakly closed; (iii) the set \(C\) is boundedly weakly compact; (iv) the operator \(P^g_C\) is norm-to-weak continuous; (iv) the operator \(P^g_C\) is maximal monotone (Theorem 4.1).