Generalized Lefschetz theorem and a fixed point index formula (Q1375160)

The Lefschetz fixed point theorem says that if the Lefschetz index \(\Lambda(f)\) of a continuous mapping \(f:X\to X\) from a finite polyhedron \(X\) into itself is non-zero, then \(f\) has a fixed point. Here, \(\Lambda(f)\) is defined as a sum \(\Lambda(f)=\sum_n (-1)^n\cdot \text{ tr}(h_n(f))\), where \(h_n(f)\) is a homomorphism of a \(Z\)-module \(B_n(X)=H_n(X)/T_n(X)\) induced by \(f\), where \(H_n(X)\) is the \(n\)-dimensional cohomology group, and \(T_n(X)\) is the torsion subgroup of \(H_n(X)\). This result can be naturally generalized to sets \(X\) which are compact ANR (absolute neighborhood retracts). In general, the Lefschetz theorem proves the existence of fixed points, but does not specify where these fixed points are. In some cases, the Lefschetz theorem can also be applied to proving the existence of a fixed point in a given subset \(C\subset X\): namely, if \(C\) is invariant under \(f\) (i.e., if \(f(C)\subseteq C\)), then we can apply the Lefschetz theorem to the restriction of \(f\) to \(C\); if the resulting index is non-zero, this means that \(f\) has a fixed point in \(C\). The paper under review shows that a generalization of the Lefschetz theorem can help in proving the existence of a fixed point in \(C\) even when \(C\) is not invariant under \(f\). For that, the author introduces the notion of a proper pair \((C,E)\) of compact sets defined by two conditions: \(C\cap f(E)\subseteq E\) and \(C\cap \overline {f(C)\backslash C}\subseteq E\) (such pairs exist for every \(C\)). For such pairs, if the appropriately defined cohomology mapping \(f_{(C,E)}\) has a non-zero Lefschetz index, then \(f\) has a fixed point in \(C\). A similar result is proven for the case when we have a sequence of proper pairs. These results are used to compute the fixed point index of a Poincaré map of a nonautonomous ordinary differential equation, and to make conclusions about the periodic solutions of such equations.