A high-order Hopf lemma for mappings into classical domains and applications (Q2142724)

The main result (Theorem 1) of the paper under review is a high-order Hopf lemma for mappings into classical domains. For an irreducible classical domain \(D\) of rank \(r\ge 1\), its boundary \(\partial D\) decomposes into \(\bigcup_{i=1}^r E_i\), where \(E_k\) lies in the closure of \(E_l\) if \(k>l\), \(E_1\) is the smooth part of \(\partial D\), and \(E_r\) is the Shilov boundary. Let \(F\) be a holomorphic map from a domain \(\Omega\) to \(D\) which extends \(C^1\)-smoothly to a smooth boundary point \(a\in \partial \Omega\). If \(F(a)\in E_k\), \(1\le k\le r\), then the main theorem says roughly that \(\rho\circ F\) vanishes to the \(k\)-th order at the point \(a\) along the normal direction (Remark 1.1). Here, \(\rho\) is the ``generic norm'' associated to the domain \(D\). To prove the main theorem, the author first establishes the characterization (Theorem 2) of \(E_k\), \(1\le k\le r\), that \[E_k=\big\{b\in \partial D: \rho\text{ vanishes to the }k\text{-th order at }b\big\}.\] Due to the distinct structures of the isotropy group \(K\subset \textup{Aut}_0(D)\) and expressions of \(\rho\), Theorems 1 and 2 are proven for the four types of classical domains case by case. In the last section, several applications of the main theorem are given, including Theorem 3 on the boundary behavior of holomorphic isometric maps from the unit ball into classical domains, Theorem 4 on the boundary behavior of holomorphic isometric maps between reducible bounded symmetric domains, and Proposition 1.1 on the transversality of proper maps into classical domains.