A necessary condition for analytic hypoellipticity (Q1901652)

The author continues his careful study of the analytic hypoellipticity of Hörmander's Laplacians. He considers here two real linearly independent analytic vector fields \(X_1\), \(X_2\) defined in an open set \(\Omega\subset \mathbb{R}^3\) such that the Hörmander's condition is satisfied (that is that the Lie algebra generated by \(X_1\) and \(X_2\) spans the tangent space to \(\mathbb{R}^3\) at each point of \(\Omega\)). Fixing any coordinate system, one can define the determinant \(\lambda(x)= \text{det}(X_1, X_2, [X_1, X_2])\). A point \(x\in \Omega\) is said to be of type 2 if \(\lambda(x)\neq 0\), and otherwise to be of higher order. A real curve \(\gamma: (- \varepsilon, \varepsilon)\mapsto \Omega\) is said to be subordinate to \(\{X_1, X_2\}\) if \(\gamma'(t)\in \text{span}\{X_1, X_2\}\) at \(\gamma(t)\) and if \(\gamma' (t)\neq 0\), for all \(|t|< \varepsilon\). One of the main results in this letter is the proof of a ``belief'' expressed by \textit{F. Trèves} [Commun. Partial Differ. Equations 3, 475-642 (1978; Zbl 0384.35055)] that the operator \(L:= X^2_1+ X^2_2\) is not analytic hypoelliptic in an open set containing a subordinate curve composed only of points of higher type. Other results concerning the \(\overline\partial_b\) are also obtained.