Discretized Gradient Flow for Manifold Learning in the Space of Embeddings

MaRDI QIDQ6313077zbMATH OpenFDO

Authors Dara Gold, Steven Rosenberg

Publication date 25 January 2019

Abstract: Gradient descent is a standard technique in machine learning to find minima of penalty functions. Many implementations of gradient descent rely on a discretized version, i.e., moving in the direction of the gradient for a set step size, recomputing the gradient, and continuing. Manifold learning/dimensionality reduction, which seeks a low dimensional manifold that best represents data in a high dimensional Euclidean space, is an inherently infinite dimensional problem. In this context, gradient descent has been applied only after simplifying manifold learning to a finite dimensional problem by e.g. RKHS or parametric methods. In this paper, we present a gradient descent approach to optimize manifold embeddings, where the gradient descent takes place in the infinite dimensional space of smooth embeddings $phi$ of a manifold $M$ into $mathbb{R}^N$. We first argue that the penalty function should be invariant under diffeomorphisms of $M$, as this guarantees that the gradient direction is always pointwise normal to $phi(M)$ inside $mathbb{R}^N$. Thus implementing discretized gradient flow in our framework requires estimating how far we can move in a fixed normal direction before leaving the space of smooth embeddings. We give an explicit lower bound for this distance in terms of the geometry of $phi(M)$ induced from its embedding in $mathbb{R}^N.$

This page was built for publication: Discretized Gradient Flow for Manifold Learning in the Space of Embeddings

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6313077)