Image Denoising: The Deep Learning Revolution and Beyond—A Survey Paper

DOI10.1137/23M1545859MaRDI QIDQ6080965zbMATH OpenFDO

Authors Michael Elad, Gregory Vaksman

Publication date 25 October 2023

Published in SIAM Journal on Imaging Sciences (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2301.03362

zbMATH Keywords

diffusion modelsimage denoisingimage synthesisinverse problemsLangevin dynamicsMMSE estimationperception-distortion trade-offperceptual qualityplug and play priorPnPREDregularization by denoising

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Computing methodologies for image processing (68U10) Image processing (compression, reconstruction, etc.) in information and communication theory (94A08) Signal theory (characterization, reconstruction, filtering, etc.) (94A12)

Abstract: Image denoising (removal of additive white Gaussian noise from an image) is one of the oldest and most studied problems in image processing. An extensive work over several decades has led to thousands of papers on this subject, and to many well-performing algorithms for this task. Indeed, 10 years ago, these achievements have led some researchers to suspect that "Denoising is Dead", in the sense that all that can be achieved in this domain has already been obtained. However, this turned out to be far from the truth, with the penetration of deep learning (DL) into image processing. The era of DL brought a revolution to image denoising, both by taking the lead in today's ability for noise removal in images, and by broadening the scope of denoising problems being treated. Our paper starts by describing this evolution, highlighting in particular the tension and synergy that exist between classical approaches and modern DL-based alternatives in design of image denoisers. The recent transitions in the field of image denoising go far beyond the ability to design better denoisers. In the 2nd part of this paper we focus on recently discovered abilities and prospects of image denoisers. We expose the possibility of using denoisers to serve other problems, such as regularizing general inverse problems and serving as the prime engine in diffusion-based image synthesis. We also unveil the idea that denoising and other inverse problems might not have a unique solution as common algorithms would have us believe. Instead, we describe constructive ways to produce randomized and diverse high quality results for inverse problems, all fueled by the progress that DL brought to image denoising. This survey paper aims to provide a broad view of the history of image denoising and closely related topics. Our aim is to give a better context to recent discoveries, and to the influence of DL in our domain.

Recommendations

• Deep learning on image denoising: an overview
• The Little Engine that Could: Regularization by Denoising (RED)
• Rate-optimal denoising with deep neural networks
• Mathematical and deep learning methods in image inverse problems
• A review of image denoising methods

Cites work

• rm K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation
• 3D Point Cloud Denoising Using Graph Laplacian Regularization of a Low Dimensional Manifold Model
• A Connection Between Score Matching and Denoising Autoencoders
• A Convergent Image Fusion Algorithm Using Scene-Adapted Gaussian-Mixture-Based Denoising
• A first-order primal-dual algorithm for convex problems with applications to imaging
• A general framework for low level vision
• A Graph Framework for Manifold-Valued Data
• A nonlocal denoising algorithm for manifold-valued images using second order statistics
• A Proximal Iteration for Deconvolving Poisson Noisy Images Using Sparse Representations
• A Review of Image Denoising Algorithms, with a New One
• A variational approach to remove outliers and impulse noise
• Adaptive wavelet thresholding for image denoising and compression
• An algorithm for total variation minimization and applications
• Annihilating Filter-Based Low-Rank Hankel Matrix Approach for Image Inpainting
• Bayesian Imaging Using Plug & Play Priors: When Langevin Meets Tweedie
• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
• Class-Aware Fully Convolutional Gaussian and Poisson Denoising
• Convolutional proximal neural networks and plug-and-play algorithms
• De-noising by soft-thresholding
• Deep Graph-Convolutional Image Denoising
• Deep learning on image denoising: an overview
• Distributed optimization and statistical learning via the alternating direction method of multipliers
• Edge-Preserving Smoothers for Image Processing
• Estimation of the mean of a multivariate normal distribution
• Exponential convergence of Langevin distributions and their discrete approximations
• Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems
• Fast interscale wavelet denoising of Poisson-corrupted images
• Fields of experts
• From Denoising to Compressed Sensing
• From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images
• Global Image Denoising
• scientific article; zbMATH DE number 3954047 (Why is no real title available?)
• scientific article; zbMATH DE number 3551792 (Why is no real title available?)
• scientific article; zbMATH DE number 1111402 (Why is no real title available?)
• scientific article; zbMATH DE number 1465030 (Why is no real title available?)
• scientific article; zbMATH DE number 802858 (Why is no real title available?)
• scientific article; zbMATH DE number 7625200 (Why is no real title available?)
• scientific article; zbMATH DE number 3349105 (Why is no real title available?)
• scientific article; zbMATH DE number 3185342 (Why is no real title available?)
• scientific article; zbMATH DE number 3046994 (Why is no real title available?)
• scientific article; zbMATH DE number 3054885 (Why is no real title available?)
• scientific article; zbMATH DE number 3105588 (Why is no real title available?)
• Ideal spatial adaptation by wavelet shrinkage
• Image Denoising in Mixed Poisson–Gaussian Noise
• Image denoising using scale mixtures of gaussians in the wavelet domain
• Image Restoration by Iterative Denoising and Backward Projections
• Image restoration via simultaneous sparse coding: where structured sparsity meets Gaussian scale mixture
• Image Selective Smoothing and Edge Detection by Nonlinear Diffusion
• Image separation using wavelets and shearlets
• Is Denoising Dead?
• Learning maximally monotone operators for image recovery
• Learning Multiscale Sparse Representations for Image and Video Restoration
• Learning representations by back-propagating errors
• Machine learning. A probabilistic perspective
• Manifold-valued image processing with SPD matrices
• Mathematical statistics. An introduction to likelihood based inference
• Nonlinear total variation based noise removal algorithms
• Nonlocal Operators with Applications to Image Processing
• Nonlocally Centralized Sparse Representation for Image Restoration
• On the local Lipschitz stability of Bayesian inverse problems
• Optimal Inversion of the Anscombe Transformation in Low-Count Poisson Image Denoising
• Patch ordering as a regularization for inverse problems in image processing
• Patch-Based Near-Optimal Image Denoising
• Plug-and-Play Unplugged: Optimization-Free Reconstruction Using Consensus Equilibrium
• Poisson noise reduction with non-local PCA
• Postprocessing of Compressed Images via Sequential Denoising
• Principal component analysis: a review and recent developments
• Principal Neighborhood Dictionaries for Nonlocal Means Image Denoising
• Priors with coupled first and second order differences for manifold-valued image processing
• Probability Theory
• Regularization by denoising via fixed-point projection (RED-PRO)
• Removal of Correlated Noise by Modeling the Signal of Interest in the Wavelet Domain
• Restoration of Poissonian Images Using Alternating Direction Optimization
• Robust principal component analysis?
• Robust Statistics
• Secrets of image denoising cuisine
• Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA)
• Solving Inverse Problems With Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity
• Sparse and redundant representations. From theory to applications in signal and image processing.
• Sparsity-Based Poisson Denoising With Dictionary Learning
• Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
• Structure-preserving deep learning
• The Little Engine that Could: Regularization by Denoising (RED)
• The statistics of natural images
• THE TRANSFORMATION OF POISSON, BINOMIAL AND NEGATIVE-BINOMIAL DATA
• Tikhonov Regularization and Total Least Squares
• Total variation regularization for manifold-valued data
• Training Products of Experts by Minimizing Contrastive Divergence
• Tweedie’s Formula and Selection Bias
• Video denoising via empirical Bayesian estimation of space-time patches
• Video Denoising, Deblocking, and Enhancement Through Separable 4-D Nonlocal Spatiotemporal Transforms

Cited in

(9)

• An alternating direction method of multipliers algorithm for the weighted fused LASSO signal approximator
• LURE: an unsupervised denoising framework for multiplicative lognormal noise
• Adversarial transferability in deep denoising models: theoretical insights and robustness enhancement via out-of-distribution typical set sampling
• A spatially variable coupled PDE-based model for image denoising using a split Bregman algorithm
• NeurTV: total variation on the neural domain
• Short communication: FISTA iterates converge linearly for denoiser-driven regularization
• Connecting image inpainting with denoising in the homogeneous diffusion setting
• Dip-based anisotropic total variation regularization for image denoising
• Plug-and-play image restoration with flow matching: a continuous viewpoint

This page was built for publication: Image Denoising: The Deep Learning Revolution and Beyond—A Survey Paper

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6080965)