A \(C^1\)-generic dichotomy for diffeomorphisms: weak forms of hyperbolicity or infinitely many sinks of sources (Q1423896)

\textit{R. Mañé} [Ann. Math. (2) 116, 503--540 (1982; Zbl 0511.58029)] proved that in the \(C^1\) topology, there is a dichotomy in possible dynamics for surface diffeomorphisms, for a residual set of diffeomorphisms. A diffeomorphism from the residual set either has infinitely many sinks or sources, or is Axiom A. An extension of this result to more dimensions is not straightforward, because of examples, for instance on three- and four-dimensional tori, of \(C^1\) robustly transitive but nonhyperbolic diffeomorphisms. The authors prove a generalization of Mañé's result replacing the notion of hyperbolicity by the existence of dominated splittings. A dominated splitting on an invariant set \(K\) for a diffeomorphism \(f\) on a manifold \(M\) is an \(f_*\) invariant splitting \(TM = E \oplus F\) with fibers of constant dimension and \[ \| f^n_* (x)| _E \| \| f^{-n}_* (f^n(x))| _F \| < 1/2 \] for \(n\) large enough. The splitting is automatically continuous. A dominated splitting on a maximal invariant set is a \(C^1\) robust property. Consider a hyperbolic saddle \(P\) for \(f\). Its homoclinic class \(H(P,f)\) is the closure of the set of transverse intersections of its stable and unstable manifolds. It is shown that for \(f\) from a residual subset of diffeomorphisms on \(M\), in the \(C^1\) topology, the following holds. Let \(P\) be a hyperbolic saddle of \(f\). Then either \(H(P,f)\) admits a dominated splitting or \(H(P,f)\) is in the closure of infinitely many sinks and sources of \(f\). As a consequence, any \(C^1\) robust transitive diffeomorphism admits a dominated splitting. A long term objective is to get a spectral decomposition theorem of the nonwandering set for nonhyperbolic diffeomorphisms away from the Newhouse phenomenon of infinitely many sinks or sources. One could for instance try to establish that the nonwandering set of diffeomorphisms away from the Newhouse phenomenon is the union of finitely many disjoint compact invariant sets admitting a dominated splitting. The authors further have some results for conservative (volume preserving) diffeomorphisms. Let \(f\) be a conservative and transitive diffeomorphism. Assume that for any nearby diffeomorphism \(g\) preserving the same volume form and every periodic point \(x = g^n(x)\), the derivative \(Dg^n(x)\) has at least one eigenvalue of modulus different from one. The authors show that then \(f\) admits a dominated splitting.