On overdetermined problems for a general class of nonlocal operators (Q2285478)

Given a continuously differentiable function \(f \colon (0,+\infty) \to \mathbb{R}\), a classical result by \textit{J. Serrin} [Arch. Ration. Mech. Anal. 43, 304--318 (1971; Zbl 0222.31007)] asserts that if there exists a solution \(u\) of the overdetermined problem \[ \begin{cases} \Delta u + f(u) = 0,\ u > 0 &\text{ in }\Omega, \\ u = 0 &\text{ on }\partial \Omega \\ u_\nu = \text{const.}& \text{ on }\partial \Omega \end{cases} \] in a smooth, bounded domain \(\Omega\subset\mathbb{R}^n\), then \(\Omega\) is a ball. Here \(u_\nu\) denotes the directional derivative of \(u\) with respect to the inward normal \(\nu\) to \(\partial\Omega\). Serrin's result was based on Aleksandrov's moving plane method, and inspired several further contributions. In particular, \textit{A.-L. Dalibard} and \textit{D. Gérard-Varet} [ESAIM, Control Optim. Calc. Var. 19, No. 4, 976--1013 (2013; Zbl 1283.49049)], followed by \textit{M. M. Fall} and the second author [ESAIM, Control Optim. Calc. Var. 21, No. 4, 924--938 (2015; Zbl 1329.35223)], studied the fractional Laplacian in bounded domains, whereas \textit{N. Soave} and \textit{E. Valdinoci} [J. Anal. Math. 137, No. 1, 101--134 (2019; Zbl 1418.35171)] investigated a similar problem in exterior and annular domains. In the paper under review, the authors consider nonlocal operators denoted by \(\Psi(-\Delta)\), where \(\Psi(t)\) is a Bernstein function, i.e., a smooth, non-negative function such that \((-1)^n\) \(d^n\Psi/dt^n \le 0\) for all \(t > 0\) and for every positive integer \(n\). Clearly, the case when \(\Psi(t) = t^s\), \(s\in (0,1)\), is included: in such a case, the operator \(\Psi(-\Delta) = (-\Delta)^s\) is the fractional Laplacian. In the case when \(D\) is a bounded (possibly disconnected) domain of class \(C^2\) in \(\mathbb{R}^d\), \(d\ge 2\), it is known that any viscosity solution \(u\in C(\mathbb{R}^d)\) of the problem \[ \Psi(-\Delta) \, u = f\text{ in }D,\quad u=0\text{ in }D^c \] can be estimated near the boundary in terms of the distance function \(\delta_D(x)\) from \(x\) to \(\partial D\): see [\textit{M. Kim} et al., J. Funct. Anal. 277, No. 1, 279--332 (2019; Zbl 1481.35102)]. More precisely, the ratio \(u(x)/V(\delta_D(x))\) is uniformly Hölder continuous in \(D\), and therefore the limit \[ \lim_{D \ni z \to x}\frac{u(z)}{V(\delta_D(z))} \] is well defined at every \(x \in \partial D\). Here the function \(V\) denotes the potential measure of a stochastic process associated to \(\Psi\). The limit above is denoted by \(\operatorname{Tr}_V(u)(x)\) (the trace of \(u/V(\delta_D)\)) and plays the role of the derivative \(u_\nu\) in Serrin's problem. In Theorem 2.2 the authors show that if the problem mentioned before has a nonnegative, nontrivial solution \(u\) satisfying \(\operatorname{Tr}_V(u)(x)=\text{constant}\geq 0\) on \(\partial D\), then \(D\) must be a ball, and \(u\) is radially symmetric and strictly decreasing in the radial direction. The case when $D$ is an exterior domain is considered afterwards. Finally, the authors also prove radial symmetry of the solution of some boundary value problems in radially symmetric domains. The proofs are based on the moving plane method. To this purpose, some maximum principles for anti-symmetric, viscosity super-solutions are established, together with a suitable version of Hopf's lemma.