Optimal transport driven CycleGAN for unsupervised learning in inverse problems

DOI10.1137/20M1317992MaRDI QIDQ5143340zbMATH OpenOpenAlexFDO

Authors Byeongsu Sim, Gyutaek Oh, Jeongsol Kim, C.-Y. Jung, Jong Chul Ye

Publication date 11 January 2021

Published in SIAM Journal on Imaging Sciences (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1909.12116

zbMATH Keywords

CycleGANinverse problemsoptimal transportpenalized least squaresunsupervised learning

Mathematics Subject Classification ID

Numerical solution to inverse problems in abstract spaces (65J22) Learning and adaptive systems in artificial intelligence (68T05) Machine vision and scene understanding (68T45) Computing methodologies for image processing (68U10)

Abstract: To improve the performance of classical generative adversarial network (GAN), Wasserstein generative adversarial networks (W-GAN) was developed as a Kantorovich dual formulation of the optimal transport (OT) problem using Wasserstein-1 distance. However, it was not clear how cycleGAN-type generative models can be derived from the optimal transport theory. Here we show that a novel cycleGAN architecture can be derived as a Kantorovich dual OT formulation if a penalized least square (PLS) cost with deep learning-based inverse path penalty is used as a transportation cost. One of the most important advantages of this formulation is that depending on the knowledge of the forward problem, distinct variations of cycleGAN architecture can be derived: for example, one with two pairs of generators and discriminators, and the other with only a single pair of generator and discriminator. Even for the two generator cases, we show that the structural knowledge of the forward operator can lead to a simpler generator architecture which significantly simplifies the neural network training. The new cycleGAN formulation, what we call the OT-cycleGAN, have been applied for various biomedical imaging problems, such as accelerated magnetic resonance imaging (MRI), super-resolution microscopy, and low-dose x-ray computed tomography (CT). Experimental results confirm the efficacy and flexibility of the theory.

Recommendations

• Optimal transport for generative models
• Multi-projection of unequal dimension optimal transport theory for generative adversary networks
• A geometric view of optimal transportation and generative model
• Supervised optimal transport
• Learning to generate Wasserstein barycenters

Cites work

• Blind image deconvolution. Methods and convergence
• Computational optimal transport. With applications to data sciences
• scientific article; zbMATH DE number 1194487 (Why is no real title available?)
• scientific article; zbMATH DE number 43443 (Why is no real title available?)
• scientific article; zbMATH DE number 936298 (Why is no real title available?)
• Optimal Transport

Cited in

(5)

• Multi-projection of unequal dimension optimal transport theory for generative adversary networks
• Low-resolution prior equilibrium network for CT reconstruction
• Clustering integrated fusion model based on the maximum matching problem
• Robustness and exploration of variational and machine learning approaches to inverse problems: an overview
• Adversarially learned iterative reconstruction for imaging inverse problems

Describes a project that uses

Uses Software

• Wasserstein GAN
• compsensing_dip
• Adam
• CycleGAN
• pix2pix
• U-Net
• QuickPALM

This page was built for publication: Optimal transport driven CycleGAN for unsupervised learning in inverse problems

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5143340)