Estimation of the density of a strongly degenerated diffusion. Study of an example (Q1261739)

In the first part of the paper the density of the 2-dimensional process \[ dx_ t =dw^ 1_ t,\quad dy_ t=g(x_ t)dw^ 2_ t+f(x_ t)dt,\quad x_ 0=x, y_ 0=y, \] is studied. Functions \(f\) and \(g\) are bounded and continuous and \(g(x)>C\exp(-C/ | x|^ \alpha)\), \(\forall x \in C^ 1\) with \(0 \leq \alpha<2\) in some neighbourhood of zero. \textit{S. Kusuoka} and \textit{D. Stroock} [J. Fac. Sci. Univ. Tokyo, Sect. I A 32, 1-76 (1985; Zbl 0568.60059)] noticed that equations of similar type may be considered with the help of the Malliavin calculus technique in the case \(f,g\in C^ \infty\) even with \(g(0)=0\), though any Hörmander type condition may fail in the origin. The main result of this part is the upper bound \(p_ t(x,y)\geq t^{-1}\exp(Ct^{- \gamma})\). The proof is based on ``direct'' estimates and various stochastic calculus tricks such as stopping times, Girsanov's transformation et al. Auxiliary results from the Malliavin calculus are used within the proof (recall that \(f\) and \(g\) may be only continuous). The existence of a smooth density along with the similar upper bounds for this density and all its derivatives are established under the additional assumption \(f,g\in C^ \infty\) with the help of the Malliavin calculus approach. Some ``pathologies on the diagonal'' are studied in the second part of the paper. Roughly, if \(g\) possesses the similar upper bound and \(f(x)>| x|^ \beta\), \(\beta>0\), then \(\forall\gamma<1\) \[ p_ t((0,0),(0,0))\leq\exp(-C(\gamma)/t^ \gamma),\;t\to 0. \] A lower bound is also established. Estimations for the density outside the diagonal of the Varadhan's large deviation type for small times are investigated in the third part. The authors use the sub-Riemannian metrics associated with the (degenerate) operator. One version does not require smoothness of \(f\) and \(g\). Stochastic analogues of \textit{E. B. Davies}' method [Heat kernels and spectral theory (1989; Zbl 0699.35006)] are proposed. Despite the word ``un exemple'' in the title, the importance of the paper seems to be much more general, though more general cases may require a new technique.