A three-operator splitting scheme and its optimization applications
From MaRDI portal
Abstract: Operator splitting schemes have been successfully used in computational sciences to reduce complex problems into a series of simpler subproblems. Since 1950s, these schemes have been widely used to solve problems in PDE and control. Recently, large-scale optimization problems in machine learning, signal processing, and imaging have created a resurgence of interest in operator-splitting based algorithms because they often have simple descriptions, are easy to code, and have (nearly) state-of-the-art performance for large-scale optimization problems. Although operator splitting techniques were introduced over 60 years ago, their importance has significantly increased in the past decade. This paper introduces a new operator-splitting scheme for solving a variety of problems that are reduced to a monotone inclusion of three operators, one of which is cocoercive. Our scheme is very simple, and it does not reduce to any existing splitting schemes. Our scheme recovers the existing forward-backward, Douglas-Rachford, and forward-Douglas-Rachford splitting schemes as special cases. Our new splitting scheme leads to a set of new and simple algorithms for a variety of other problems, including the 3-set split feasibility problems, 3-objective minimization problems, and doubly and multiple regularization problems, as well as the simplest extension of the classic ADMM from 2 to 3 blocks of variables. In addition to the basic scheme, we introduce several modifications and enhancements that can improve the convergence rate in practice, including an acceleration that achieves the optimal rate of convergence for strongly monotone inclusions. Finally, we evaluate the algorithm on several applications.
Recommendations
- A new splitting method for monotone inclusions of three operators
- Asymmetric forward-backward-adjoint splitting for solving monotone inclusions involving three operators
- Convergence analysis of an inexact three-operator splitting algorithm
- Three-operator splitting algorithm for a class of variational inclusion problems
- A unified splitting algorithm for composite monotone inclusions
Cites work
- scientific article; zbMATH DE number 3574917 (Why is no real title available?)
- scientific article; zbMATH DE number 3108780 (Why is no real title available?)
- A Douglas--Rachford Type Primal-Dual Method for Solving Inclusions with Mixtures of Composite and Parallel-Sum Type Monotone Operators
- A Modified Forward-Backward Splitting Method for Maximal Monotone Mappings
- A dual algorithm for the solution of nonlinear variational problems via finite element approximation
- A first-order primal-dual algorithm for convex problems with applications to imaging
- A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms
- A splitting algorithm for dual monotone inclusions involving cocoercive operators
- Applications of a Splitting Algorithm to Decomposition in Convex Programming and Variational Inequalities
- Backward-forward algorithms for structured monotone inclusions in Hilbert spaces
- Compositions and convex combinations of averaged nonexpansive operators
- Convergence Rate Analysis of Primal-Dual Splitting Schemes
- Convergence rate analysis of several splitting schemes
- Convergence rate analysis of the forward-Douglas-Rachford splitting scheme
- Convex analysis and monotone operator theory in Hilbert spaces
- Coordinate-friendly structures, algorithms and applications
- Ergodic convergence to a zero of the sum of monotone operators in Hilbert space
- Mean Value Methods in Iteration
- On the convergence rate improvement of a primal-dual splitting algorithm for solving monotone inclusion problems
- On weak convergence of the Douglas-Rachford method
- Primal-dual splitting algorithm for solving inclusions with mixtures of composite, Lipschitzian, and parallel-sum type monotone operators
- Splitting Algorithms for the Sum of Two Nonlinear Operators
- Systems of Structured Monotone Inclusions: Duality, Algorithms, and Applications
- The rate of linear convergence of the Douglas-Rachford algorithm for subspaces is the cosine of the Friedrichs angle
Cited in
(only showing first 100 items - show all)- Iteration complexity of an inexact Douglas-Rachford method and of a Douglas-Rachford-Tseng's F-B four-operator splitting method for solving monotone inclusions
- Dualize, split, randomize: toward fast nonsmooth optimization algorithms
- Four-operator splitting via a forward-backward-half-forward algorithm with line search
- Recovery of a time-dependent bottom topography function from the shallow water equations via an adjoint approach
- MiKM: multi-step inertial Krasnosel'skiǐ-Mann algorithm and its applications
- Projective splitting with forward steps
- Convergence analysis of an inexact three-operator splitting algorithm
- A Variable Metric Forward-Reflected-Douglas-Rachford Method for Solving Monotone Inclusions
- Novel forward-backward algorithms for optimization and applications to compressive sensing and image inpainting
- Uniqueness of DRS as the 2 operator resolvent-splitting and impossibility of 3 operator resolvent-splitting
- Faster subgradient methods for functions with Hölderian growth
- A two-level distributed algorithm for nonconvex constrained optimization
- Inertial-relaxed splitting for composite monotone inclusions
- Distributed continuous-time proximal algorithm for nonsmooth resource allocation problem with coupled constraints
- Dual descent augmented Lagrangian method and alternating direction method of multipliers
- An envelope for Davis-Yin splitting and strict saddle-point avoidance
- Convergence rates of forward-Douglas-Rachford splitting method
- Monotone operator theory in convex optimization
- Regularization proximal method for monotone variational inclusions
- Splitting with near-circulant linear systems: applications to total variation CT and PET
- Direct covariance matrix estimation with compositional data
- Convergence Rates for Projective Splitting
- Estimation of graphical models through structured norm minimization
- Inexact alternating direction methods of multipliers for separable convex optimization
- Multi-step inertial forward-backward-half forward algorithm for solving monotone inclusion
- Proximal variable smoothing method for three-composite nonconvex nonsmooth minimization with a linear operator
- Primal dual methods for Wasserstein gradient flows
- Converting ADMM to a proximal gradient for efficient sparse estimation
- A linear algebra perspective on the random multi-block ADMM: the QP case
- New convergence analysis of a primal-dual algorithm with large stepsizes
- A parameterized three-operator splitting algorithm for non-convex minimization problems with applications
- On the optimal relaxation parameters of Krasnosel'ski–Mann iteration
- Envelope functions: unifications and further properties
- Asynchronous parallel primal-dual block coordinate update methods for affinely constrained convex programs
- Learning to optimize: a tutorial for continuous and mixed-integer optimization
- Proximal-based recursive implementation for model-free data-driven fault diagnosis
- Distributed forward-backward methods for ring networks
- Self-adaptive forward-backward splitting algorithm for the sum of two monotone operators in Banach spaces
- Hybrid Jacobian and Gauss-Seidel proximal block coordinate update methods for linearly constrained convex programming
- Efficient alternating minimization methods for variational edge-weighted colorization models
- An inertial semi-forward-reflected-backward splitting and its application
- PET-MRI joint reconstruction with common edge weighted total variation regularization
- A three-operator splitting algorithm with deviations for generalized DC programming
- An accelerated forward-backward-half forward splitting algorithm for monotone inclusion with applications to image restoration
- A Three-Operator Splitting Perspective of a Three-Block ADMM for Convex Quadratic Semidefinite Programming and Beyond
- Generalized Kalman smoothing: modeling and algorithms
- A fixed-point proximity algorithm for recovering low-rank components from incomplete observation data with application to motion capture data refinement
- Forward-reflected-backward splitting algorithms with momentum: weak, linear and strong convergence results
- Three-operator splitting for learning to predict equilibria in convex games
- [[:Publication:6091103|Fast Krasnosel’skiĭ–Mann Algorithm with a Convergence Rate of the Fixed Point Iteration of \(\boldsymbol{{ o} \left(\frac{1}Template:K \right)}\)]]
- Alternating direction method of multipliers for nonconvex log total variation image restoration
- Convergence and rate analysis of a proximal linearized ADMM for nonconvex nonsmooth optimization
- Stochastic projective splitting
- On unbounded delays in asynchronous parallel fixed-point algorithms
- A coordinate-descent primal-dual algorithm with large step size and possibly nonseparable functions
- Solving monotone inclusions involving the sum of three maximally monotone operators and a cocoercive operator with applications
- On the efficiency of random permutation for ADMM and coordinate descent
- A unified convergence rate analysis of the accelerated smoothed gap reduction algorithm
- Three-operator reflected forward-backward splitting algorithm with double inertial effects
- Extrapolated plug-and-play three-operator splitting methods for nonconvex optimization with applications to image restoration
- Approximate first-order primal-dual algorithms for saddle point problems
- Linear convergence of primal-dual gradient methods and their performance in distributed optimization
- A dynamical splitting method for minimizing the sum of three convex functions
- Asymmetric forward-backward-adjoint splitting for solving monotone inclusions involving three operators
- Operator splitting performance estimation: tight contraction factors and optimal parameter selection
- Convergence rates for an inexact ADMM applied to separable convex optimization
- The geometry of monotone operator splitting methods
- Forward-reflected-backward splitting method without cocoercivity for the sum of maximal monotone operators in Banach spaces
- A first-order inexact primal-dual algorithm for a class of convex-concave saddle point problems
- Backward-forward-reflected-backward splitting for three operator monotone inclusions
- Two-step fixed-point proximity algorithms for multi-block separable convex problems
- Cyclic coordinate-update algorithms for fixed-point problems: analysis and applications
- Multi-step inertial Krasnosel'skiǐ-Mann iteration with new inertial parameters arrays
- Proximal Splitting Algorithms for Convex Optimization: A Tour of Recent Advances, with New Twists
- Single-forward-step projective splitting: exploiting cocoercivity
- A technique with diminishing and non-summable step-size for monotone inclusion problems in Banach spaces
- A second-order adaptive Douglas-Rachford dynamic method for maximal \(\alpha\)-monotone operators
- Finding the forward-Douglas-Rachford-forward method
- On the equivalence of the primal-dual hybrid gradient method and Douglas-Rachford splitting
- Strengthened splitting methods for computing resolvents
- From Halpern's fixed-point iterations to Nesterov's accelerated interpretations for root-finding problems
- On the ergodic convergence rates of a first-order primal-dual algorithm
- On the convergence rate of the Krasnosel'skiĭ-Mann iteration
- Reflected three-operator splitting method for monotone inclusion problem
- Fejér-monotone hybrid steepest descent method for affinely constrained and composite convex minimization
- Two new splitting methods for three-operator monotone inclusions in Hilbert spaces
- Tight coefficients of averaged operators via scaled relative graph
- ADMM for monotone operators: convergence analysis and rates
- A new primal-dual algorithm for minimizing the sum of three functions with a linear operator
- Modified forward-backward splitting method for variational inclusions
- Relaxed forward-backward splitting methods for solving variational inclusions and applications
- Convergence analysis of the direct extension of ADMM for multiple-block separable convex minimization
- An Efficient Convex Formulation for Reduced-Rank Linear Discriminant Analysis in High Dimensions
- A direct proof of convergence of Davis-Yin splitting algorithm allowing larger stepsizes
- A product space reformulation with reduced dimension for splitting algorithms
- Three-operator splitting algorithm for a class of variational inclusion problems
- On the Global Linear Convergence of the ADMM with MultiBlock Variables
- A forward-backward splitting method for monotone inclusions without cocoercivity
- Iterative regularization methods with new stepsize rules for solving variational inclusions
- Nonlinear forward-backward splitting with momentum correction
This page was built for publication: A three-operator splitting scheme and its optimization applications
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q683303)
