Spencer cohomologies and symmetry groups (Q1903564)

Let \(\alpha : E (\alpha) \to M\) be a linear bundle, let a subbundle \(r\) in its \(k \)th order jet prolongation \(\alpha^k\) represent a formally integrable system of differential equations on the sections of \(\alpha\). Consider the Spencer complex \(0 \to {\mathcal R}^{(l)} \to {\mathcal R}^{ (l - 1)} \otimes \Lambda^1 (M) \to \cdots\) and the corresponding stable Spencer homology \({\mathcal H}^i (r)\). In the situation when the manifold \(M\) is fibred over another manifold \(B\) and the projection \(\kappa : M \to B\) is ``noncharacteristic'', the authors introduce a filtration in the Spencer complex such that its associated spectral sequence converges to Spencer cohomology of \(r\) and the term \(E^{pq}_2\) is isomorphic to the de Rham cohomology of the manifold \(B\) with coefficients in what is called a ``fiberwise Spencer cohomology'' and is denoted by \({\mathcal H}^\bullet_\kappa (r)\). This construction naturally generalizes the Leray-Serre spectral sequence and reduces to it when the system \(r\) is determined by the de Rham operator \(d : C^\infty M \to \Lambda^1 M\). Next, the authors consider a compact Lie group \(G\) and its representation by automorphisms of the bundle \(\alpha\) leaving the system \(r\) invariant. Another representation arises in the second (sophisticated) Spencer complex \(0 \to C^0 \to C^1 \to \cdots\) (the stable one). Its isotype decomposition is compatible with the differential, giving rise to a decomposition of Spencer cohomology groups \({\mathcal H}^q (R) = \oplus_{\mu \in Ch (G)} {\mathcal H}^q_\mu (R)\), \(Ch (G)\) being the character group. It turns out (theorem of coincidence) that in the majority of cases \({\mathcal H}^q_\mu (R) = 0\) for all nontrivial characters \(\mu\) so that \({\mathcal H}^q (R)\) coincides with the ``invariant'' Spencer cohomology \({\mathcal H}^q_e (R)\) \((e\) denoting the trivial character) [cf. the authors, Russ. Acad. Sci., Dokl., Math. 47, No. 1, 122-125 (1993); translation from Dokl. Akad. Nauk, Ross. Akad. Nauk 328, No. 5, 544-546 (1993; Zbl 0818.58051)]. The Appendix deals with cohomology computations for complexes equipped with a set of anticommuting differentials.