iPiano: Inertial Proximal Algorithm for Nonconvex Optimization

DOI10.1137/130942954zbMath1296.90094arXiv1404.4805OpenAlexW2000462146MaRDI QIDQ2876847

Thomas Pock, Peter Ochs, Thomas Brox, Yunjin Chen

Publication date: 20 August 2014

Published in: SIAM Journal on Imaging Sciences (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1404.4805

zbMATH Keywords

nonconvex optimizationKurdyka-Łojasiewicz inequalityproof of convergenceheavy-ball methodinertial forward-backward splitting

Mathematics Subject Classification ID

Large-scale problems in mathematical programming (90C06) Nonconvex programming, global optimization (90C26) Nonlinear programming (90C30) Numerical optimization and variational techniques (65K10) Methods of quasi-Newton type (90C53) Pattern recognition, speech recognition (68T10) Image processing (compression, reconstruction, etc.) in information and communication theory (94A08)

Related Items

An inertial proximal alternating direction method of multipliers for nonconvex optimization ⋮ Penalty schemes with inertial effects for monotone inclusion problems ⋮ Convex-Concave Backtracking for Inertial Bregman Proximal Gradient Algorithms in Nonconvex Optimization ⋮ A General Non-Lipschitz Infimal Convolution Regularized Model: Lower Bound Theory and Algorithm ⋮ On an iteratively reweighted linesearch based algorithm for nonconvex composite optimization ⋮ Convergence of the Momentum Method for Semialgebraic Functions with Locally Lipschitz Gradients ⋮ Convergence Analysis of the Proximal Gradient Method in the Presence of the Kurdyka–Łojasiewicz Property Without Global Lipschitz Assumptions ⋮ Unnamed Item ⋮ Inertial projected gradient method for large-scale topology optimization ⋮ A new Lagrangian-based first-order method for nonconvex constrained optimization ⋮ Heavy-ball-based hard thresholding algorithms for sparse signal recovery ⋮ An inertial alternating minimization with Bregman distance for a class of nonconvex and nonsmooth problems ⋮ Nested alternating minimization with FISTA for non-convex and non-smooth optimization problems ⋮ A class of modified accelerated proximal gradient methods for nonsmooth and nonconvex minimization problems ⋮ On Proximal Algorithms with Inertial Effects Beyond Monotonicity ⋮ A unified surface geometric framework for feature-aware denoising, hole filling and context-aware completion ⋮ A refined inertial DC algorithm for DC programming ⋮ Convergence of Random Reshuffling under the Kurdyka–Łojasiewicz Inequality ⋮ A forward-backward algorithm with different inertial terms for structured non-convex minimization problems ⋮ An extrapolated proximal iteratively reweighted method for nonconvex composite optimization problems ⋮ A proximal trust-region method for nonsmooth optimization with inexact function and gradient evaluations ⋮ Calculus rules of the generalized concave Kurdyka-Łojasiewicz property ⋮ Smoothing fast proximal gradient algorithm for the relaxation of matrix rank regularization problem ⋮ On and Beyond Total Variation Regularization in Imaging: The Role of Space Variance ⋮ Fast optimization via inertial dynamics with closed-loop damping ⋮ Proximal gradient method with extrapolation and line search for a class of non-convex and non-smooth problems ⋮ Double inertial projection method for variational inequalities with quasi-monotonicity ⋮ On inertial iterated Tikhonov methods for solving ill-posed problems ⋮ An accelerated forward-backward-half forward splitting algorithm for monotone inclusion with applications to image restoration ⋮ Convergence of Inexact Forward--Backward Algorithms Using the Forward--Backward Envelope ⋮ Unifying Abstract Inexact Convergence Theorems and Block Coordinate Variable Metric iPiano ⋮ A Fokker-Planck Based Approach to Control Jump Processes ⋮ Unnamed Item ⋮ Acceleration and Global Convergence of a First-Order Primal-Dual Method for Nonconvex Problems ⋮ Blind image fusion for hyperspectral imaging with the directional total variation ⋮ On the Edge Recovery Property of Noncovex Nonsmooth Regularization in Image Restoration ⋮ An introduction to continuous optimization for imaging ⋮ Inertial Variable Metric Techniques for the Inexact Forward--Backward Algorithm ⋮ A New Optimization Approach to Sparse Reconstruction of Log-Conductivity in Acousto-Electric Tomography ⋮ Composite Optimization by Nonconvex Majorization-Minimization ⋮ Modern regularization methods for inverse problems ⋮ Adaptive FISTA for Nonconvex Optimization ⋮ Variational Image Regularization with Euler's Elastica Using a Discrete Gradient Scheme ⋮ Scalable Bayesian Uncertainty Quantification in Imaging Inverse Problems via Convex Optimization ⋮ On a general smoothly truncated regularization for variational piecewise constant image restoration: construction and convergent algorithms ⋮ The Primal-Dual Hybrid Gradient Method for Semiconvex Splittings ⋮ Choose Your Path Wisely: Gradient Descent in a Bregman Distance Framework ⋮ Learnable Descent Algorithm for Nonsmooth Nonconvex Image Reconstruction ⋮ Distributed Stochastic Inertial-Accelerated Methods with Delayed Derivatives for Nonconvex Problems ⋮ Self-Adaptive Inertial Projection and Contraction Algorithm for Monotone Variational Inequality ⋮ An accelerated majorization-minimization algorithm with convergence guarantee for non-Lipschitz wavelet synthesis model ^* ⋮ Extrapolated Proximal Subgradient Algorithms for Nonconvex and Nonsmooth Fractional Programs ⋮ The Exact Modulus of the Generalized Concave Kurdyka-Łojasiewicz Property ⋮ Optimization on Spheres: Models and Proximal Algorithms with Computational Performance Comparisons ⋮ The lower bound of nonlocal gradient for non-convex and non-smooth image patches based regularization ⋮ Pseudo-inverses of difference matrices and their application to sparse signal approximation ⋮ Bregman proximal gradient algorithms for deep matrix factorization ⋮ Forward-backward quasi-Newton methods for nonsmooth optimization problems ⋮ Convergence results of a new monotone inertial forward-backward splitting algorithm under the local Hölder error bound condition ⋮ Continuous dynamics related to monotone inclusions and non-smooth optimization problems ⋮ An inertial forward-backward algorithm for the minimization of the sum of two nonconvex functions ⋮ Block-coordinate and incremental aggregated proximal gradient methods for nonsmooth nonconvex problems ⋮ Inertial Douglas-Rachford splitting for monotone inclusion problems ⋮ A General Inertial Proximal Point Algorithm for Mixed Variational Inequality Problem ⋮ Convergence of inexact quasisubgradient methods with extrapolation ⋮ Variational methods for normal integration ⋮ Malitsky-Tam forward-reflected-backward splitting method for nonconvex minimization problems ⋮ Field of experts regularized nonlocal low rank matrix approximation for image denoising ⋮ Local convergence of the heavy-ball method and iPiano for non-convex optimization ⋮ An inertial Bregman generalized alternating direction method of multipliers for nonconvex optimization ⋮ Self-adaptive inertial subgradient extragradient scheme for pseudomonotone variational inequality problem ⋮ Inertial Proximal ADMM for Linearly Constrained Separable Convex Optimization ⋮ Limiting Aspects of Nonconvex ${TV}^{\phi}$ Models ⋮ A fast proximal iteratively reweighted nuclear norm algorithm for nonconvex low-rank matrix minimization problems ⋮ Poisson Noise Reduction with Higher-Order Natural Image Prior Model ⋮ Inertial Proximal Alternating Linearized Minimization (iPALM) for Nonconvex and Nonsmooth Problems ⋮ A forward-backward dynamical approach to the minimization of the sum of a nonsmooth convex with a smooth nonconvex function ⋮ Techniques for gradient-based bilevel optimization with non-smooth lower level problems ⋮ Global convergence of model function based Bregman proximal minimization algorithms ⋮ Approaching nonsmooth nonconvex optimization problems through first order dynamical systems with hidden acceleration and Hessian driven damping terms ⋮ An inertial Tseng's type proximal algorithm for nonsmooth and nonconvex optimization problems ⋮ New acceleration factors of the Krasnosel'skiĭ-Mann iteration ⋮ A multi-parameter regularization model for deblurring images corrupted by impulsive noise ⋮ New convergence results for the inexact variable metric forward-backward method ⋮ Inertial proximal alternating minimization for nonconvex and nonsmooth problems ⋮ Global convergence of proximal iteratively reweighted algorithm ⋮ Separating variables to accelerate non-convex regularized optimization ⋮ Forward-Backward Envelope for the Sum of Two Nonconvex Functions: Further Properties and Nonmonotone Linesearch Algorithms ⋮ iPiasco: inertial proximal algorithm for strongly convex optimization ⋮ Nonconvex mixed TV/Cahn-Hilliard functional for super-resolution/segmentation of 3D trabecular bone images ⋮ General inertial proximal gradient method for a class of nonconvex nonsmooth optimization problems ⋮ Iteratively reweighted \(\ell _1\) algorithms with extrapolation ⋮ Non-smooth non-convex Bregman minimization: unification and new algorithms ⋮ Generalized forward-backward splitting with penalization for monotone inclusion problems ⋮ A Symmetric Alternating Direction Method of Multipliers for Separable Nonconvex Minimization Problems ⋮ A proximal quasi-Newton method based on memoryless modified symmetric rank-one formula ⋮ Bregman Itoh-Abe methods for sparse optimisation ⋮ Ultrasound image restoration based on a learned dictionary and a higher-order MRF ⋮ The Shannon total variation ⋮ Inertial projection and contraction algorithms for variational inequalities ⋮ An abstract convergence framework with application to inertial inexact forward-backward methods ⋮ A smoothing proximal gradient algorithm with extrapolation for the relaxation of \({\ell_0}\) regularization problem ⋮ Geodesic PCA versus Log-PCA of Histograms in the Wasserstein Space ⋮ Speckle reduction with trained nonlinear diffusion filtering ⋮ Unnamed Item ⋮ Optimisation of classic photometric stereo by non-convex variational minimisation ⋮ Bilevel Optimization with Nonsmooth Lower Level Problems ⋮ Inertial hybrid splitting methods for operator inclusion problems ⋮ Acceleration of the imaginary time method for spectrally computing the stationary states of Gross-Pitaevskii equations ⋮ Convergence rates for an inertial algorithm of gradient type associated to a smooth non-convex minimization ⋮ Approximate versions of proximal iteratively reweighted algorithms including an extended IP-ICMM for signal and image processing problems ⋮ A general truncated regularization framework for contrast-preserving variational signal and image restoration: motivation and implementation ⋮ An inertial subgradient-type method for solving single-valued variational inequalities and fixed point problems ⋮ Approaching nonsmooth nonconvex minimization through second-order proximal-gradient dynamical systems ⋮ Gradient-type penalty method with inertial effects for solving constrained convex optimization problems with smooth data ⋮ An inertial proximal-gradient penalization scheme for constrained convex optimization problems ⋮ On the proximal gradient algorithm with alternated inertia ⋮ Efficient Learning with a Family of Nonconvex Regularizers by Redistributing Nonconvexity ⋮ Proximal primal-dual best approximation algorithm with memory ⋮ Inertial alternating generalized forward-backward splitting for image colorization ⋮ On the discontinuity of images recovered by noncovex nonsmooth regularized isotropic models with box constraints ⋮ Linear convergence of inexact descent method and inexact proximal gradient algorithms for lower-order regularization problems ⋮ Inertial proximal gradient methods with Bregman regularization for a class of nonconvex optimization problems ⋮ Momentum and stochastic momentum for stochastic gradient, Newton, proximal point and subspace descent methods ⋮ Calculus of the exponent of Kurdyka-Łojasiewicz inequality and its applications to linear convergence of first-order methods ⋮ An inertial algorithm for DC programming ⋮ A block coordinate variable metric forward-backward algorithm ⋮ A block coordinate variable metric linesearch based proximal gradient method ⋮ An efficient variational method for restoring images with combined additive and multiplicative noise ⋮ Whiteness constraints in a unified variational framework for image restoration ⋮ A logarithmic image prior for blind deconvolution ⋮ A block inertial Bregman proximal algorithm for nonsmooth nonconvex problems with application to symmetric nonnegative matrix tri-factorization ⋮ Inertial iterative algorithms for common solution of variational inequality and system of variational inequalities problems ⋮ A Variable Metric Forward-Backward Method with Extrapolation ⋮ iPiano ⋮ A Gauss-Seidel type inertial proximal alternating linearized minimization for a class of nonconvex optimization problems ⋮ An accelerated smoothing gradient method for nonconvex nonsmooth minimization in image processing ⋮ A new proximal iterative hard thresholding method with extrapolation for \(\ell _0\) minimization ⋮ An inexact proximal gradient algorithm with extrapolation for a class of nonconvex nonsmooth optimization problems ⋮ The extragradient algorithm with inertial effects for solving the variational inequality ⋮ Block-proximal methods with spatially adapted acceleration ⋮ Application of a new accelerated algorithm to regression problems ⋮ A general double-proximal gradient algorithm for d.c. programming ⋮ An inertial proximal partially symmetric ADMM-based algorithm for linearly constrained multi-block nonconvex optimization problems with applications ⋮ On the geometric analysis of a quartic-quadratic optimization problem under a spherical constraint ⋮ Two-step inertial Bregman alternating minimization algorithm for nonconvex and nonsmooth problems ⋮ An adaptive Polyak heavy-ball method ⋮ The proximity operator of the log-sum penalty ⋮ Minimization of non-smooth, non-convex functionals by iterative thresholding ⋮ Extrapolated smoothing descent algorithm for constrained nonconvex and nonsmooth composite problems ⋮ Splitting methods with variable metric for Kurdyka-Łojasiewicz functions and general convergence rates ⋮ On damped second-order gradient systems ⋮ Generalized fractional algebraic linear system solvers ⋮ Inertial projection and contraction algorithms with larger step sizes for solving quasimonotone variational inequalities ⋮ An inertial forward-backward-forward primal-dual splitting algorithm for solving monotone inclusion problems