Generalized 3G theorem and application to relativistic stable process on non-smooth open sets (Q883509)

If \(k\in (0,1/2]\), a \(k\)-fat \(D\subset{\mathbb R}^{d}\) is defined as an open one for which there exists \(R>0\) such that, for every \(Q\in \partial D\) and \(r\in (0,R)\), there exists \(A_{r}(Q)\in D\) with the ball \(B(A_{r}(Q),kr)\) \(\subset D\cap B(Q,r)\). The generalized 3G theorem (main result of the paper) is proved in section 3, after 16 lemmas. It involves a symmetric \(\alpha\)-stable process on \({\mathbb R}^{d}\), \(d\geq 2\), i.e. a Lévy process having a transition density \(p(t,y-x)\) with respect to the Lebesgue measure, satisfying \(\int e^{ix\cdot \xi }p(t,x)\,dx= e^{-t| \xi | ^{\alpha }}\) and states that, if \(D\) is a bounded \(k\)-fat set, then there exist \(c\) and \(\gamma <\alpha\) such that, for the Green function \(G_{D}\) of the process killed at \({\mathbb C}D\), \(G_{D}(x,y)G_{D}(z,w)/G_{D}(x,w)\) is less than the product of \(c\), of \((1\vee ((| x-w| \wedge | y-z| )/| x-y| ))^{\gamma }\), of \((1\vee ((| x-w| \wedge | y-z| )/| z-w| ))^{\gamma }\) and of \(| x-w| ^{d-\alpha }| x-y| ^{\alpha -d}| z-w| ^{\alpha -d}\), \(\vee\), \(\wedge\) meaning max, min, respectively. The 3G theorem is applied in section 4. If \(H\) is the set of all singular harmonic \(h\geq 0\) for \(X\) in \(D\) and we define \(E_{x}^{h}(f(X_{t}^{D}))=E_{x}(f(X_{t}^{D})h(X_{t}^{D})/h(x))\). etc., then, for \(\beta >\alpha\), \(\sup_{x\in D,h\in H}E_{x}^{h}(\sum_{s<\tau_{D}}| X_{s-}^{D}- X_{s}^{D}| ^{\beta })<\infty\) (1), \(\tau_{D}\) being the moment of the first exit of \(X\) from \(D\). (1) is valid also for \(h\) running over \(G_{D}(\cdot ,w)\), \(w\in D\). The relation involving \(H\) relies on the 3G inequality for the Martin kernel \(M_{D}\), \(x,y,z\in D,w\in \partial D\). Then (also in section 4) the relativistic \(\alpha\)-stable process \(X^{m}\) in \({\mathbb R}^{d}\), \(\alpha \in (0,2)\), is considered: \(E(\exp(i\xi \cdot (X_{t}^{m}-X_{0}^{m})))= \exp(-t((| \xi | ^{2}+m^{2/\alpha })^{\alpha / 2}-m))\), \(m>0\). The authors prove that there exists \(V_{D}(x,y)\), \(x,y\in D\), jointly continuous outside the diagonal, with \(\int V_{D}(x,y)f(y)\,dy =E_{x}(\int_0^{\tau_D} f(X_{s}^{m})\,ds)\), that \(V_{D}/G_{D}\in [c^{-1},c]\) for some constant \(c>0\) and the 3G inequality for \(V_{D}\), generalize (1) to \(V_{D}(\cdot ,w)\), show the existence of \(K_{D}(x,w)= \lim_{y\rightarrow w,y\in D}V_{D}(x,y)/V_{D}(x_{0},y)\) for \(w\in \partial D,x\in D\), that \(M_{D}/K_{D}\in [c'{}^{-1},c']\), the 3G theorem for the Martin kernel \(K_{D}\) with \(w\in \partial D\), the representation of every singular \(X^{m}\) harmonic \(u\geq 0\) on \(D\) as \(\int_{\partial D} K_{D}(x ,\cdot )\,d\nu\) for a finite measure \(\nu\) on \(\partial D\), generalize (1) to \(X^{m}\) and its \(h\)'s and show that, if \(Q\in \partial D,x\rightarrow Q\), \(| x-Q| <\beta \operatorname{dist}(y,\partial D),\beta >(1-k)/k\), then \(\lim(v(x)/u(x))\) exists \(\nu\)-a.e., if \(v\) is ``another \(u\)''. The proofs rely on Dirichlet forms, Lévy kernels, a property of \(F\) with \(| F(x,y)| \leq c| x-y| ^{\beta }\), \(\beta >\alpha\) involving the Lévy kernel and \(G_{D}\). In the last paragraph (5), generalizations to symmetric Markov processes are indicated. The first two paragraphs contain the history of the problem, motivations for generalizing the 3G theorem, statements of the two used results, from the papers [\textit{K. Bogdan}, Stud. Math. 123, No. 1, 43--80 (1997; Zbl 0870.31009) and \textit{R. Song} and \textit{J.-M. Wu}, J. Funct. Anal 168, No. 2, 403--427 (1999; Zbl 0945.31006)], figuring among the References.