Improving ``fast iterative shrinkage-thresholding algorithm: faster, smarter, and greedier
From MaRDI portal
Publication:5075693
Abstract: The "fast iterative shrinkage-thresholding algorithm", a.k.a. FISTA, is one of the most well-known first-order optimisation scheme in the literature, as it achieves the worst-case optimal convergence rate in terms of objective function value. However, despite such an optimal theoretical convergence rate, in practice the (local) oscillatory behaviour of FISTA often damps its efficiency. Over the past years, various efforts are made in the literature to improve the practical performance of FISTA, such as monotone FISTA, restarting FISTA and backtracking strategies. In this paper, we propose a simple yet effective modification to the original FISTA scheme which has two advantages: it allows us to 1) prove the convergence of generated sequence; 2) design a so-called "lazy-start" strategy which can up to an order faster than the original scheme. Moreover, by exploring the properties of FISTA scheme, we propose novel adaptive and greedy strategies which probes the limit of the algorithm. The advantages of the proposed schemes are tested through problems arising from inverse problem, machine learning and signal/image processing.
Recommendations
- scientific article; zbMATH DE number 7753352
- Some modified fast iterative shrinkage thresholding algorithms with a new adaptive non-monotone stepsize strategy for nonsmooth and convex minimization problems
- Over-relaxation of the fast iterative shrinkage-thresholding algorithm with variable stepsize
- Another look at the fast iterative shrinkage/thresholding algorithm (FISTA)
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
Cites work
- scientific article; zbMATH DE number 3850830 (Why is no real title available?)
- scientific article; zbMATH DE number 4164577 (Why is no real title available?)
- scientific article; zbMATH DE number 3790208 (Why is no real title available?)
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
- Activity identification and local linear convergence of forward-backward-type methods
- Adaptive restart for accelerated gradient schemes
- Adaptive restart of the optimized gradient method for convex optimization
- An inertial forward-backward algorithm for monotone inclusions
- An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping
- An introduction to continuous optimization for imaging
- Backtracking strategies for accelerated descent methods with smooth composite objectives
- Convergence of a splitting inertial proximal method for monotone operators
- Convergence rates of forward-Douglas-Rachford splitting method
- Convergence rates of inertial forward-backward algorithms
- Convergence rates with inexact non-expansive operators
- Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity
- Inertial forward-backward algorithms with perturbations: application to Tikhonov regularization
- Inertial, corrected, primal-dual proximal splitting
- Introductory lectures on convex optimization. A basic course.
- Nonlinear total variation based noise removal algorithms
- On the convergence of the iterates of the ``fast iterative shrinkage/thresholding algorithm
- Robust principal component analysis?
- Signal Recovery by Proximal Forward-Backward Splitting
- Some methods of speeding up the convergence of iteration methods
- Splitting Algorithms for the Sum of Two Nonlinear Operators
- The rate of convergence of Nesterov's accelerated forward-backward method is actually faster than \(1/k^2\)
- Weak convergence of the sequence of successive approximations for nonexpansive mappings
Cited in
(22)- Parameter-free FISTA by adaptive restart and backtracking
- Another look at the fast iterative shrinkage/thresholding algorithm (FISTA)
- On the convergence of the iterates of the ``fast iterative shrinkage/thresholding algorithm
- scientific article; zbMATH DE number 7753352 (Why is no real title available?)
- ``FISTA in Banach spaces with adaptive discretisations
- Computable centering methods for spiraling algorithms and their duals, with motivations from the theory of Lyapunov functions
- New proximal type algorithms for convex minimization and its application to image deblurring
- Scaled, inexact, and adaptive generalized FISTA for strongly convex optimization
- Forward-reflected-backward splitting algorithms with momentum: weak, linear and strong convergence results
- Applying FISTA to optimization problems (with or) without minimizers
- A class of modified accelerated proximal gradient methods for nonsmooth and nonconvex minimization problems
- A fast inertial primal-dual algorithm to composite optimization models with application to image restoration problems
- Some modified fast iterative shrinkage thresholding algorithms with a new adaptive non-monotone stepsize strategy for nonsmooth and convex minimization problems
- Approximation method for monotone inclusion problems in real Banach spaces with applications
- Polynomial preconditioners for regularized linear inverse problems
- Fast gradient method for low-rank matrix estimation
- On FISTA with a relative error rule
- Over-relaxation of the fast iterative shrinkage-thresholding algorithm with variable stepsize
- Linearly-convergent FISTA variant for composite optimization with duality
- Improving first-order threshold implementations of \textsf{SKINNY}
- FISTA is an automatic geometrically optimized algorithm for strongly convex functions
- An improved parameterized fast iterative shrinkage-thresholding algorithm with adaptive step size and its applications
This page was built for publication: Improving ``fast iterative shrinkage-thresholding algorithm: faster, smarter, and greedier
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5075693)
