Modes of homogeneous gradient flows

DOI10.1137/20M1388577MaRDI QIDQ5860347zbMATH OpenOpenAlexFDO

Authors Ido Cohen, Omri Azencot, Pavel Lifshits, Guy Gilboa

Publication date 19 November 2021

Published in SIAM Journal on Imaging Sciences (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2007.01534

zbMATH Keywords

dynamic mode decompositiongradient flowshomogeneous operatorsnonlinear decompositionnonlinear spectral theory

Mathematics Subject Classification ID

Nonlinear eigenvalue problems and nonlinear spectral theory for PDEs (35P30) Decomposition methods (49M27) Spectral, collocation and related methods for initial value and initial-boundary value problems involving PDEs (65M70) Signal theory (characterization, reconstruction, filtering, etc.) (94A12)

Abstract: Finding latent structures in data is drawing increasing attention in diverse fields such as image and signal processing, fluid dynamics, and machine learning. In this work we examine the problem of finding the main modes of gradient flows. Gradient descent is a fundamental process in optimization where its stochastic version is prominent in training of neural networks. Here our aim is to establish a consistent theory for gradient flows $ps{i}_t=P(psi)$, where $P$ is a nonlinear homogeneous operator. Our proposed framework stems from analytic solutions of homogeneous flows, previously formalized by Cohen-Gilboa, where the initial condition $ps{i}_0$ admits the nonlinear eigenvalue problem $P(ps{i}_0)=lambdaps{i}_0$. We first present an analytic solution for ac{DMD} in such cases. We show an inherent flaw of ac{DMD}, which is unable to recover the essential dynamics of the flow. It is evident that ac{DMD} is best suited for homogeneous flows of degree one. We propose an adaptive time sampling scheme and show its dynamics are analogue to homogeneous flows of degree one with a fixed step size. Moreover, we adapt ac{DMD} to yield a real spectrum, using symmetric matrices. Our analytic solution of the proposed scheme recovers the dynamics perfectly and yields zero error. We then proceed to show that in the general case the orthogonal modes $ph{i}_i$ are approximately nonlinear eigenfunctions $P(ph{i}_i)approxlambd{a}_{i}ph{i}_i$. We formulate Orthogonal Nonlinear Spectral decomposition (emph{OrthoNS}), which recovers the essential latent structures of the gradient descent process. Definitions for spectrum and filtering are given, and a Parseval-type identity is shown.

Recommendations

• Total-variation mode decomposition
• Consistent Dynamic Mode Decomposition
• Spectral analysis of nonlinear flows
• Variants of dynamic mode decomposition: boundary condition, Koopman, and Fourier analyses
• Flows generating nonlinear eigenfunctions

Cites work

• A data-driven approximation of the koopman operator: extending dynamic mode decomposition
• A Hessenberg-Schur method for the problem AX + XB= C
• A spectral approach to total variation
• A Total Variation Spectral Framework for Scale and Texture Analysis
• Algorithm 432 [C2]: Solution of the matrix equation AX + XB = C [F4]
• Analysis and synthesis of stability matrices
• Consistent Dynamic Mode Decomposition
• Convergence of entropic schemes for optimal transport and gradient flows
• Dynamic mode decomposition of numerical and experimental data
• Energy dissipating flows for solving nonlinear eigenpair problems
• Entropic Burgers' equation via a minimizing movement scheme based on the Wasserstein metric
• Extended dynamic mode decomposition with dictionary learning: A data-driven adaptive spectral decomposition of the Koopman operator
• Gradient flows in metric spaces and in the space of probability measures
• scientific article; zbMATH DE number 6678650 (Why is no real title available?)
• scientific article; zbMATH DE number 1012640 (Why is no real title available?)
• scientific article; zbMATH DE number 1111402 (Why is no real title available?)
• scientific article; zbMATH DE number 803211 (Why is no real title available?)
• scientific article; zbMATH DE number 3398324 (Why is no real title available?)
• Linear algebra and learning from data
• Matrix calculations for Liapunov quadratic forms
• Nonlinear Differential Equations of Monotone Types in Banach Spaces
• Nonlinear eigenproblems in image processing and computer vision
• Nonlinear spectral decompositions by gradient flows of one-homogeneous functionals
• Nonlinear total variation based noise removal algorithms
• Optimization methods for large-scale machine learning
• Randomized Dynamic Mode Decomposition
• Solution of the Equation AX + XB = C by Inversion of an M \times M or N \times N Matrix
• Solution of the matrix equation \(AX-XB=C\)
• Spectral decompositions using one-homogeneous functionals
• Spectral properties of dynamical systems, model reduction and decompositions
• Stability by Liapunov's direct method. With applications
• The Finite Harmonic Oscillator and Its Applications to Sequences, Communication, and Radar
• The Optimal Hard Threshold for Singular Values is <inline-formula> <tex-math notation="TeX">\(4/\sqrt {3}\) </tex-math></inline-formula>
• Variable projection methods for an optimized dynamic mode decomposition

Cited in

(3)

• Multiplicative dynamic mode decomposition
• Functional dimensionality of Koopman eigenfunction space
• Total-variation mode decomposition

Describes a project that uses

Uses Software

• SinGAN
• LaplacianSmoothing-GradientDescent
• SpectralNet

This page was built for publication: Modes of homogeneous gradient flows

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5860347)