Integration by parts formula for regional fractional Laplacian (Q881637)

The generator of a symmetric \(\alpha\)-stable process is the fractional Laplacian \(-(-\Delta)^{\alpha/2}\). As this operator is non-local, it cannot be restricted to a domain automatically. Restricting the corresponding integral kernel leads to a jump-process on a domain \(G\) -- called censored stable process. The corresponding generator \(\Delta_G^{\alpha/2}\) -- the so-called regional fractional Laplacian on \(G\) -- is given by \[ \Delta_G^{\alpha/2} u(x):=\lim_{\varepsilon\downarrow 0} C_{n,\alpha} \int_{y \in G, | y-x| >\varepsilon} \frac{u(y)-u(x)}{| x-y| ^{n+\alpha}}\,dy, \quad x\in G, \] provided the limit exists. In the present paper, the author establishes the integration by parts formula for \(\Delta_G^{\alpha/2}\). This generalizes the by now established integration by parts formula for fractional derivatives for open subintervals of the real line on the one hand and the Gauss-Green formula for the classical Laplacian on bounded \(C^2\)-smooth open sets on the other hand. This involves in particular the definition of the (fractional) boundary operator (generalizing the notion of normal derivative) as well as the introduction of suitable function spaces. Results are given for \(\alpha \in (1,2)\) as well as for \(\alpha \in (0,1]\). Moreover, a similar integration by parts formula for more general non local operators is obtained.