Parallel Selective Algorithms for Nonconvex Big Data Optimization

DOI10.1109/TSP.2015.2399858zbMath1394.94174arXiv1402.5521OpenAlexW2058361915MaRDI QIDQ4580494

Gesualdo Scutari, Francisco Facchinei, Simone Sagratella

Publication date: 22 August 2018

Published in: IEEE Transactions on Signal Processing (Search for Journal in Brave)

Abstract: We propose a decomposition framework for the parallel optimization of the sum of a differentiable (possibly nonconvex) function and a (block) separable nonsmooth, convex one. The latter term is usually employed to enforce structure in the solution, typically sparsity. Our framework is very flexible and includes both fully parallel Jacobi schemes and Gauss- Seidel (i.e., sequential) ones, as well as virtually all possibilities "in between" with only a subset of variables updated at each iteration. Our theoretical convergence results improve on existing ones, and numerical results on LASSO, logistic regression, and some nonconvex quadratic problems show that the new method consistently outperforms existing algorithms.

Full work available at URL: https://arxiv.org/abs/1402.5521

Mathematics Subject Classification ID

Nonconvex programming, global optimization (90C26) Signal theory (characterization, reconstruction, filtering, etc.) (94A12)

Related Items (24)

Feasible methods for nonconvex nonsmooth problems with applications in green communications ⋮ Decentralized Dictionary Learning Over Time-Varying Digraphs ⋮ A flexible coordinate descent method ⋮ Distributed Optimization Based on Gradient Tracking Revisited: Enhancing Convergence Rate via Surrogation ⋮ Unnamed Item ⋮ Distributed algorithms for convex problems with linear coupling constraints ⋮ Combining approximation and exact penalty in hierarchical programming ⋮ A framework for parallel and distributed training of neural networks ⋮ Scalable subspace methods for derivative-free nonlinear least-squares optimization ⋮ Distributed semi-supervised support vector machines ⋮ An adaptive partial linearization method for optimization problems on product sets ⋮ A framework for parallel second order incremental optimization algorithms for solving partially separable problems ⋮ Asynchronous Stochastic Coordinate Descent: Parallelism and Convergence Properties ⋮ Distributed nonconvex constrained optimization over time-varying digraphs ⋮ Distributed optimization methods for nonconvex problems with inequality constraints over time-varying networks ⋮ Localization and approximations for distributed non-convex optimization ⋮ Asynchronous parallel algorithms for nonconvex optimization ⋮ Computing B-Stationary Points of Nonsmooth DC Programs ⋮ Unnamed Item ⋮ A Fast Active Set Block Coordinate Descent Algorithm for $\ell_1$-Regularized Least Squares ⋮ Parallel decomposition methods for linearly constrained problems subject to simple bound with application to the SVMs training ⋮ Ghost Penalties in Nonconvex Constrained Optimization: Diminishing Stepsizes and Iteration Complexity ⋮ Unnamed Item ⋮ On the solution of monotone nested variational inequalities

This page was built for publication: Parallel Selective Algorithms for Nonconvex Big Data Optimization