A unified convergence analysis of block successive minimization methods for nonsmooth optimization
From MaRDI portal
Abstract: The block coordinate descent (BCD) method is widely used for minimizing a continuous function f of several block variables. At each iteration of this method, a single block of variables is optimized, while the remaining variables are held fixed. To ensure the convergence of the BCD method, the subproblem to be optimized in each iteration needs to be solved exactly to its unique optimal solution. Unfortunately, these requirements are often too restrictive for many practical scenarios. In this paper, we study an alternative inexact BCD approach which updates the variable blocks by successively minimizing a sequence of approximations of f which are either locally tight upper bounds of f or strictly convex local approximations of f. We focus on characterizing the convergence properties for a fairly wide class of such methods, especially for the cases where the objective functions are either non-differentiable or nonconvex. Our results unify and extend the existing convergence results for many classical algorithms such as the BCD method, the difference of convex functions (DC) method, the expectation maximization (EM) algorithm, as well as the alternating proximal minimization algorithm.
Recommendations
- Block coordinate proximal gradient methods with variable Bregman functions for nonsmooth separable optimization
- Iteration complexity analysis of block coordinate descent methods
- On the linear convergence of the approximate proximal splitting method for non-smooth convex optimization
- Block-coordinate and incremental aggregated proximal gradient methods for nonsmooth nonconvex problems
- Synchronous parallel block coordinate descent method for nonsmooth convex function minimization
Cited in
(only showing first 100 items - show all)- Stream-suitable optimization algorithms for some soft-margin support vector machine variants
- A convergent least-squares regularized blind deconvolution approach
- Visualizing the effects of a changing distance on data using continuous embeddings
- Linear mixed models with marginally symmetric nonparametric random effects
- Maximum likelihood estimation of triangular and polygonal distributions
- A globally convergent algorithm for Lasso-penalized mixture of linear regression models
- A globally convergent algorithm for nonconvex optimization based on block coordinate update
- Extended ADMM and BCD for nonseparable convex minimization models with quadratic coupling terms: convergence analysis and insights
- Large-scale unit commitment under uncertainty: an updated literature survey
- A cyclic block coordinate descent method with generalized gradient projections
- Asynchronous parallel primal-dual block coordinate update methods for affinely constrained convex programs
- On the pervasiveness of difference-convexity in optimization and statistics
- Pathwise coordinate optimization for sparse learning: algorithm and theory
- DC programming and DCA: thirty years of developments
- Tensor completion using total variation and low-rank matrix factorization
- A block coordinate variable metric linesearch based proximal gradient method
- Toward fast transform learning
- An inexact PAM method for computing Wasserstein barycenter with unknown supports
- Improving the accuracy of LDG approximations on coarse meshes
- 3D joint hydrogeophysical inversion using similarity measures
- Linear convergence of inexact descent method and inexact proximal gradient algorithms for lower-order regularization problems
- An efficient low complexity algorithm for box-constrained weighted maximin dispersion problem
- Algorithms for nonnegative matrix factorization with the Kullback-Leibler divergence
- Multi-block Bregman proximal alternating linearized minimization and its application to orthogonal nonnegative matrix factorization
- An efficient alternating minimization method for fourth degree polynomial optimization
- The regularized feasible directions method for nonconvex optimization
- A novel update rule of HALS algorithm for nonnegative matrix factorization and Zangwill's global convergence
- Sequential support points
- Multiview clustering via exclusive non-negative subspace learning and constraint propagation
- Nonparametric mean-lower partial moment model and enhanced index investment
- Inertial alternating direction method of multipliers for non-convex non-smooth optimization
- Group collaborative representation for image set classification
- Matrix factorization for low-rank tensor completion using framelet prior
- Randomized mixture models for probability density approximation and estimation
- Worst-case complexity of cyclic coordinate descent: O(n^2) gap with randomized version
- Variable metric techniques for forward-backward methods in imaging
- Optimality conditions and DC-Dinkelbach-type algorithm for generalized fractional programs with ratios of difference of convex functions
- On the superiority of PGMs to PDCAs in nonsmooth nonconvex sparse regression
- Empirical Bayesian learning in AR graphical models
- Hyperspectral image restoration using framelet-regularized low-rank nonnegative matrix factorization
- Low-rank tensor completion via smooth matrix factorization
- Block-simultaneous direction method of multipliers: a proximal primal-dual splitting algorithm for nonconvex problems with multiple constraints
- Low-rank tensor completion using matrix factorization based on tensor train rank and total variation
- Coordinate descent algorithms
- Large-scale unit commitment under uncertainty
- On the convergence of inexact block coordinate descent methods for constrained optimization
- An expectation-maximization algorithm for blind separation of noisy mixtures using Gaussian mixture model
- Maximum likelihood estimation of Gaussian mixture models without matrix operations
- Perturbed proximal primal-dual algorithm for nonconvex nonsmooth optimization
- Distributed constraint-coupled optimization via primal decomposition over random time-varying graphs
- A survey on deep matrix factorizations
- Error bound and isocost imply linear convergence of DCA-based algorithms to D-stationarity
- Wideband MIMO radar transmit beampattern synthesis via majorization-minimization
- Laplace mixture autoregressive models
- Block stochastic gradient iteration for convex and nonconvex optimization
- Computing B-stationary points of nonsmooth DC programs
- A stochastic successive minimization method for nonsmooth nonconvex optimization with applications to transceiver design in wireless communication networks
- A proximal strictly contractive Peaceman-Rachford splitting method for convex programming with applications to imaging
- A block successive upper-bound minimization method of multipliers for linearly constrained convex optimization
- Iterative Proportional Scaling Revisited: A Modern Optimization Perspective
- Block coordinate proximal gradient methods with variable Bregman functions for nonsmooth separable optimization
- A proximal block minimization method of multipliers with a substitution procedure
- On faster convergence of cyclic block coordinate descent-type methods for strongly convex minimization
- Hybrid Jacobian and Gauss-Seidel proximal block coordinate update methods for linearly constrained convex programming
- Decomposition methods for computing directional stationary solutions of a class of nonsmooth nonconvex optimization problems
- On the linear convergence of the approximate proximal splitting method for non-smooth convex optimization
- A remark on accelerated block coordinate descent for computing the proximity operators of a sum of convex functions
- A unified framework for structured graph learning via spectral constraints
- Alternating minimization, scaling algorithms, and the null-cone problem from invariant theory
- Learning optimal distributionally robust individualized treatment rules
- Ghost penalties in nonconvex constrained optimization: diminishing stepsizes and iteration complexity
- Outlier accommodation in moving-horizon state estimation: a risk-averse performance-specified approach
- Splitting proximal algorithms for convex optimizations over metric spaces with curvature bounded above
- Block Bregman majorization minimization with extrapolation
- Asynchronous variance-reduced block schemes for composite non-convex stochastic optimization: block-specific steplengths and adapted batch-sizes
- Maximizing convergence time in network averaging dynamics subject to edge removal
- scientific article; zbMATH DE number 7626710 (Why is no real title available?)
- A First-Order Optimization Algorithm for Statistical Learning with Hierarchical Sparsity Structure
- scientific article; zbMATH DE number 7409363 (Why is no real title available?)
- GAITA: a Gauss-Seidel iterative thresholding algorithm for _q regularized least squares regression
- Iteratively reweighted group Lasso based on log-composite regularization
- The trimmed Lasso: sparse recovery guarantees and practical optimization by the generalized soft-min penalty
- On the Convergence to Stationary Points of Deterministic and Randomized Feasible Descent Directions Methods
- An inexact variable metric proximal point algorithm for generic quasi-Newton acceleration
- A simple framework for stability analysis of state-dependent networks of heterogeneous agents
- The dynamics of swamps in the canonical tensor approximation problem
- Incremental majorization-minimization optimization with application to large-scale machine learning
- Iteration complexity analysis of block coordinate descent methods
- Cyclic coordinate-update algorithms for fixed-point problems: analysis and applications
- A block successive lower-bound maximization algorithm for the maximum pseudo-likelihood estimation of fully visible Boltzmann machines
- Maximum pseudolikelihood estimation for model-based clustering of time series data
- Alternating proximal regularized dictionary learning
- A generic coordinate descent solver for non-smooth convex optimisation
- Convergence of a block coordinate descent method for nondifferentiable minimization
- Scalable Collaborative Ranking for Personalized Prediction
- A proximal alternating minimization algorithm for the largest C-eigenvalue of piezoelectric-type tensors
- Spatial clustering of time series via mixture of autoregressions models and Markov random fields
- The convergence properties of infeasible inexact proximal alternating linearized minimization
- On real structured controllability/stabilizability/stability radius: complexity and unified rank-relaxation based methods
- Local linear convergence of proximal coordinate descent algorithm
This page was built for publication: A unified convergence analysis of block successive minimization methods for nonsmooth optimization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2848188)
