A singular Riemannian geometry approach to deep neural networks. I: Theoretical foundations (Q6077759)

In this interesting paper under review, the authors propose a theoretical framework which can be employed to study neural networks. Equipping the output manifold \(M'\) with a Riemannian metric \(g\) and computing its pullback to the input manifold \(M\), they obtain a singular Riemannian metric on \(M\), which in turn induces a pseudo-distance \(d\) between input points. A first property shows that the pseudodistance between two input points that are mapped by the network to the same output is zero. Then the authors consider the Kolmogorov quotient of \(M\) by \(d\), establishing that the resulting space is of the same dimension as \(M'\) and that the pullback of the Riemannian metric on \(M'\) remains a Riemannian metric, provided that the full rank hypothesis is assumed. It is also proved that with respect to this quotient all the input points mapped to the same output belong to the same equivalence class. Moreover, the singular Riemannian metric induces a natural decomposition of the tangent bundle of \(M\) as a direct sum of a vertical and a horizontal bundle, the latter being an integrable distribution. This decomposition allows the authors to employ the singular Riemannian metric of \(M\) to characterize the equivalence classes of the quotient space.