Invertible residual networks in the context of regularization theory for linear inverse problems

DOI10.1088/1361-6420/AD0660arXiv2306.01335MaRDI QIDQ6141568FDOQ6141568

Sören Dittmer, Clemens Arndt, Judith Nickel, Alexander Denker, Tobias Kluth, Author name not available (Why is that?), Author name not available (Why is that?), Peter Maass

Publication date: 20 December 2023

Published in: Inverse Problems (Search for Journal in Brave)

Abstract: Learned inverse problem solvers exhibit remarkable performance in applications like image reconstruction tasks. These data-driven reconstruction methods often follow a two-step scheme. First, one trains the often neural network-based reconstruction scheme via a dataset. Second, one applies the scheme to new measurements to obtain reconstructions. We follow these steps but parameterize the reconstruction scheme with invertible residual networks (iResNets). We demonstrate that the invertibility enables investigating the influence of the training and architecture choices on the resulting reconstruction scheme. For example, assuming local approximation properties of the network, we show that these schemes become convergent regularizations. In addition, the investigations reveal a formal link to the linear regularization theory of linear inverse problems and provide a nonlinear spectral regularization for particular architecture classes. On the numerical side, we investigate the local approximation property of selected trained architectures and present a series of experiments on the MNIST dataset that underpin and extend our theoretical findings.

Full work available at URL: https://arxiv.org/abs/2306.01335

zbMATH Keywords

local approximation propertyinvertible residual networksconvergent regularizationlearned nonlinear spectral regularizationlearning for inverse problems

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Numerical solutions of ill-posed problems in abstract spaces; regularization (65J20) Numerical solution to inverse problems in abstract spaces (65J22)

Cites Work

• Title not available (Why is that?)
• Regularization methods in Banach spaces.
• Title not available (Why is that?)
• How general are general source conditions?
• Data-Driven Nonsmooth Optimization
• Designing Optimal Spectral Filters for Inverse Problems
• Deep Convolutional Neural Network for Inverse Problems in Imaging
• NETT: solving inverse problems with deep neural networks
• Solving inverse problems using data-driven models
• CLIP: cheap Lipschitz training of neural networks
• Modern regularization methods for inverse problems
• Regularization by architecture: a deep prior approach for inverse problems
• Deep null space learning for inverse problems: convergence analysis and rates
• Regularization theory of the analytic deep prior approach
• Regularization of inverse problems by filtered diagonal frame decomposition
• Learning Maximally Monotone Operators for Image Recovery
• Bayesian Imaging Using Plug & Play Priors: When Langevin Meets Tweedie
• Data driven regularization by projection

Cited In (4)

• Regularization by architecture: a deep prior approach for inverse problems
• Fixed-point algorithms for inverse of residual rectifier neural networks
• Neural-network-based regularization methods for inverse problems in imaging
• Convergence of non-linear diagonal frame filtering for regularizing inverse problems

This page was built for publication: Invertible residual networks in the context of regularization theory for linear inverse problems

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6141568)