A Hopf's lemma and the boundary regularity for the fractional \(p\)-Laplacian (Q1725807)

The paper deals with pointwise solutions, sometimes called \textit{classical} solutions, of problems associated to the fractional $p$-Laplacian, which is the operator $(-\Delta)^s_p$ given by \[ (-\Delta)^s_p \, u(x) = C_{n,s,p}\, \mathrm{PV}\int_{\mathbb R^n} \frac{|u(x) - u(y)|^{p - 2} \, (u(x) - u(y))}{|x - y|^{n + sp}} \, dy, \] where PV denotes Cauchy's principal value, $s \in (0,1)$, $p \in (1,+\infty)$, and $C_{n,s,p}$ is a suitable constant. In order that the definition is well posed, the authors consider sufficiently smooth functions $u : \mathbb R^n \to \mathbb R$, $n \ge 1$, such that \[ \int_{\mathbb R^n}\frac{|u(x)|^{p - 1}}{1 + |x|^{n + sp}}\, dx<+\infty. \] The first result in the paper is an extension to the fractional $p$-Laplacian, with $p \ge 3$, of a Hopf-type lemma whose proof is outlined by \textit{C. Li} and \textit{W. Chen} for the fractional Laplacian ($p = 2$) in [Proc. Am. Math. Soc. 147, 1565--1575 (2019; Zbl 1417.35034)]. The result applies to the anti-symmetric function $w_\lambda(x) = u_\lambda(x) - u(x)$, where $u_\lambda(x) = u(x^\lambda)$ and $x^\lambda$ is the symmetric point to $x$ with respect to a given hyperplane $T_\lambda$. The authors assume that $w_\lambda$ is positive in one of the two half-spaces, say $\Sigma_\lambda$, bounded by $T_\lambda$, and $u,u_\lambda$ and $w_\lambda$ satisfy \[ (-\Delta)^s_p \, u_\lambda(x) - (-\Delta)^s_p \, u(x) + c(x)\,w_\lambda(x) = 0\text{ in }\Sigma_\lambda, \] where the coefficient $c(x)$ is subject to the same condition as in [\textit{W. Chen} and \textit{C. Li}, ``A Hopf type lemma for fractional equations'', Preprint, \url{arXiv:1705.04889}, Theorem 1]. They show that the outward derivative of $w_\lambda$ at $T_\lambda$ (which is twice the inward derivative of $u$ at $T_\lambda$) is different from zero. The second result in the paper deals with the Dirichlet problem \[ (-\Delta)^s_p \, u(x) = f(x)\text{ in }\Omega,\quad u = 0\text{ in }\mathbb R^n \setminus \Omega, \] where $p > 2$, $f$ is bounded, and $\Omega$ is a bounded domain satisfying the exterior sphere condition. The $C^\alpha(\overline \Omega)$-regularity of $u$ is investigated for $p \in (1,+\infty)$ by \textit{A. Iannizzotto} et al. [Rev. Mat. Iberoam. 32, No. 4, 1353--1392 (2016; Zbl 1433.35447)]. Inspired by the paper [\textit{A. Iannizzotto} et al., Rev. Mat. Iberoam. 32, No. 4, 1353--1392 (2016; Zbl 1433.35447)] and by the book [\textit{W. Chen} et al., The fractional Laplacian (to appear). Hackensack, NJ: World Scientific (2020; Zbl 1451.35002)], the authors prove that for every exponent $\nu \in (0,s)$ there exists a convenient constant $c$ such that \[ |u(x)| \leq c\,(\operatorname{dist}(x,\partial\Omega))^\nu . \] The proof is based on the construction of a suitable barrier in an annular region bounded by two concentric spherical surfaces. The following example is provided by the reviewer in order to clarify the mutual relationship between local Hölder continuity, uniform Hölder continuity, and the inequality above. More specifically, the non-negative function \[ \varphi(x) = \left| x \, \sin\frac\pi{\, x^2 \,} \right|^\frac12 , \] defined in the open interval $\Omega = (0,1) \subset \mathbb R$, belongs to the Hölder class $C^\frac12(\Omega)$ and satisfies the inequality $\varphi(x) \leq c \, (\operatorname{dist}(x,\partial\Omega))^\frac12$ with a suitable constant $c$. However, if we let $x_k = k^{-\frac12}$ and $y_k = (k + \frac12)^{-\frac12}$, then the sequence \[ \frac{|\varphi(y_k) - \varphi(x_k)|}{|y_k - x_k|^\frac12} \] approaches $+\infty$ as $k \to +\infty$, hence $\varphi$ does not belong to $C^\frac12(\overline \Omega)$.