Improving ``fast iterative shrinkage-thresholding algorithm: faster, smarter, and greedier
From MaRDI portal
Publication:5075693
Abstract: The "fast iterative shrinkage-thresholding algorithm", a.k.a. FISTA, is one of the most well-known first-order optimisation scheme in the literature, as it achieves the worst-case optimal convergence rate in terms of objective function value. However, despite such an optimal theoretical convergence rate, in practice the (local) oscillatory behaviour of FISTA often damps its efficiency. Over the past years, various efforts are made in the literature to improve the practical performance of FISTA, such as monotone FISTA, restarting FISTA and backtracking strategies. In this paper, we propose a simple yet effective modification to the original FISTA scheme which has two advantages: it allows us to 1) prove the convergence of generated sequence; 2) design a so-called "lazy-start" strategy which can up to an order faster than the original scheme. Moreover, by exploring the properties of FISTA scheme, we propose novel adaptive and greedy strategies which probes the limit of the algorithm. The advantages of the proposed schemes are tested through problems arising from inverse problem, machine learning and signal/image processing.
Recommendations
- scientific article; zbMATH DE number 7753352
- Some modified fast iterative shrinkage thresholding algorithms with a new adaptive non-monotone stepsize strategy for nonsmooth and convex minimization problems
- Over-relaxation of the fast iterative shrinkage-thresholding algorithm with variable stepsize
- Another look at the fast iterative shrinkage/thresholding algorithm (FISTA)
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
Cites work
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
- Activity identification and local linear convergence of forward-backward-type methods
- Adaptive restart for accelerated gradient schemes
- Adaptive restart of the optimized gradient method for convex optimization
- An inertial forward-backward algorithm for monotone inclusions
- An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping
- An introduction to continuous optimization for imaging
- Backtracking strategies for accelerated descent methods with smooth composite objectives
- Convergence of a splitting inertial proximal method for monotone operators
- Convergence rates of forward-Douglas-Rachford splitting method
- Convergence rates of inertial forward-backward algorithms
- Convergence rates with inexact non-expansive operators
- Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity
- scientific article; zbMATH DE number 3850830 (Why is no real title available?)
- scientific article; zbMATH DE number 4164577 (Why is no real title available?)
- scientific article; zbMATH DE number 3790208 (Why is no real title available?)
- Inertial forward-backward algorithms with perturbations: application to Tikhonov regularization
- Inertial, corrected, primal-dual proximal splitting
- Introductory lectures on convex optimization. A basic course.
- Nonlinear total variation based noise removal algorithms
- On the convergence of the iterates of the ``fast iterative shrinkage/thresholding algorithm
- Robust principal component analysis?
- Signal Recovery by Proximal Forward-Backward Splitting
- Some methods of speeding up the convergence of iteration methods
- Splitting Algorithms for the Sum of Two Nonlinear Operators
- The rate of convergence of Nesterov's accelerated forward-backward method is actually faster than 1/k^2
- Weak convergence of the sequence of successive approximations for nonexpansive mappings
Cited in
(32)- Some modified fast iterative shrinkage thresholding algorithms with a new adaptive non-monotone stepsize strategy for nonsmooth and convex minimization problems
- New proximal type algorithms for convex minimization and its application to image deblurring
- Approximation method for monotone inclusion problems in real Banach spaces with applications
- Applying FISTA to optimization problems (with or) without minimizers
- On FISTA with a relative error rule
- Over-relaxation of the fast iterative shrinkage-thresholding algorithm with variable stepsize
- Another look at the fast iterative shrinkage/thresholding algorithm (FISTA)
- On the convergence of the iterates of the ``fast iterative shrinkage/thresholding algorithm
- Scaled, inexact, and adaptive generalized FISTA for strongly convex optimization
- ``FISTA in Banach spaces with adaptive discretisations
- Computable centering methods for spiraling algorithms and their duals, with motivations from the theory of Lyapunov functions
- scientific article; zbMATH DE number 7753352 (Why is no real title available?)
- Linearly-convergent FISTA variant for composite optimization with duality
- Fast gradient method for low-rank matrix estimation
- FISTA is an automatic geometrically optimized algorithm for strongly convex functions
- A class of modified accelerated proximal gradient methods for nonsmooth and nonconvex minimization problems
- Improving first-order threshold implementations of \textsf{SKINNY}
- A fast inertial primal-dual algorithm to composite optimization models with application to image restoration problems
- Polynomial preconditioners for regularized linear inverse problems
- Forward-reflected-backward splitting algorithms with momentum: weak, linear and strong convergence results
- Parameter-free FISTA by adaptive restart and backtracking
- An improved parameterized fast iterative shrinkage-thresholding algorithm with adaptive step size and its applications
- Accelerated forward-backward algorithm based on inertial and correction terms with linesearch for solving convex minimization problem and its application
- A separate preconditioned primal-dual splitting algorithm for composite monotone inclusion problems
- Training of extreme learning machine network based on novel generalized inertial forward-reflected-backward splitting algorithm
- An adaptive stochastic gradient algorithm based on FISTA for convex optimization
- Practical proximal primal-dual algorithms for structured saddle point problems
- Enhancing convergence speed in sparse signal denoising: the APFISTA algorithm
- Efficient parameter-free restarted accelerated gradient methods for convex and strongly convex optimization
- Convergence and applications of novel FISTA-like algorithms
- A double inertial viscosity-type method for inclusion problems with fixed point constraints
- Enhancing efficiency of proximal gradient method with predicted and corrected step sizes
This page was built for publication: Improving ``fast iterative shrinkage-thresholding algorithm: faster, smarter, and greedier
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5075693)
