Rigidity of Hawking mass for surfaces in three manifolds (Q1684496)

It was proved by \textit{D. Christodoulou} and \textit{S. T. Yau} [Contemp. Math. 71, 9--14 (1988; Zbl 0685.53050)] that the Hawking mass of a stable CMC sphere in a three-manifold with nonnegative scalar curvature is nonnegative. In this paper, the author gives a partial answer to the rigidity problem of Hawking mass, which was proposed by \textit{R. Bartnik} [in: Proceedings of the international congress of mathematicians, ICM 2002, Beijing, China, August 20--28, 2002. Vol. II: Invited lectures. Beijing: Higher Education Press; Singapore: World Scientific/distributor. 231--240 (2002; Zbl 1009.53055)] and was stated as: What can we say about the ambient manifold if the Hawking mass vanished for some surface? Firstly, the author proves that if a surface \(\Sigma=\partial\Omega\) in a complete three-manifold with nonnegative scalar curvature is a nearly round stable CMC sphere with zero Hawking mass, then the domain \(\Omega\) enclosed by \(\Sigma\) is an Euclidean ball in \(\mathbb{R}^3\). This result is also true in the hyperbolic case. This is a local flatness result and we can only expect the flatness inside the surface. This is reasonable since \textit{A. Carlotto} and \textit{R. Schoen} [Invent. Math. 205, No. 3, 559--615 (2016; Zbl 1353.83010)] constructed examples of three-manifolds with nonnegative scalar curvature which are flat in half space of \(\mathbb{R}^3\). The proof is by transforming the rigidity problem to a mean-field type equation and proving the uniqueness result near the zero solution. Secondly, the author proves the following global flatness result: If there exists a nearly round isoperimetric sphere with zero Hawking mass in a complete asymptotically flat (AF) three-manifold \((M,g)\) with nonnegative scalar curvature, then \((M,g)\) is isometric to the Euclidean space. Similarly, the hyperbolic version of this result also holds. This can be used to say that a complete AF three-manifold with nonnegative scalar curvature having a large isoperimetric surface with zero Hawking mass must be flat. For the rigidity of small isoperimetric surfaces, the author also proves that if there exists a small isoperimetric surface \(\Sigma\) with zero maximal Hawking mass (which is defined as the Hawking mass of the surface which achieves the minimal mean curvature enclosing the same volume of \(\Sigma\)) in a complete AF three-manifold \((M,g)\) with nonnegative scalar curvature and having no compact minimal surface, then \((M,g)\) is flat. The proof relies on the monotonicity of the Hawking mass and the fact that the topology of a small isoperimetric surface is a sphere.