Parallel Selective Algorithms for Nonconvex Big Data Optimization

DOI10.1109/TSP.2015.2399858MaRDI QIDQ4580494zbMATH OpenOpenAlexFDO

Authors Francisco Facchinei, Gesualdo Scutari, Simone Sagratella

Publication date 22 August 2018

Published in IEEE Transactions on Signal Processing (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1402.5521

Mathematics Subject Classification ID

Nonconvex programming, global optimization (90C26) Signal theory (characterization, reconstruction, filtering, etc.) (94A12)

Abstract: We propose a decomposition framework for the parallel optimization of the sum of a differentiable (possibly nonconvex) function and a (block) separable nonsmooth, convex one. The latter term is usually employed to enforce structure in the solution, typically sparsity. Our framework is very flexible and includes both fully parallel Jacobi schemes and Gauss- Seidel (i.e., sequential) ones, as well as virtually all possibilities "in between" with only a subset of variables updated at each iteration. Our theoretical convergence results improve on existing ones, and numerical results on LASSO, logistic regression, and some nonconvex quadratic problems show that the new method consistently outperforms existing algorithms.

Cited in

(25)

• A flexible coordinate descent method
• An adaptive partial linearization method for optimization problems on product sets
• Distributed optimization methods for nonconvex problems with inequality constraints over time-varying networks
• Parallel decomposition methods for linearly constrained problems subject to simple bound with application to the SVMs training
• On the solution of monotone nested variational inequalities
• Distributed algorithms for convex problems with linear coupling constraints
• A framework for parallel and distributed training of neural networks
• Asynchronous parallel algorithms for nonconvex optimization
• Feasible methods for nonconvex nonsmooth problems with applications in green communications
• Distributed semi-supervised support vector machines
• A framework for parallel second order incremental optimization algorithms for solving partially separable problems
• Distributed nonconvex constrained optimization over time-varying digraphs
• Parallel coordinate descent methods for big data optimization
• A fast active set block coordinate descent algorithm for _1-regularized least squares
• Asynchronous stochastic coordinate descent: parallelism and convergence properties
• Computing B-stationary points of nonsmooth DC programs
• A distributed block coordinate descent method for training l₁ regularized linear classifiers
• A class of parallel doubly stochastic algorithms for large-scale learning
• Ghost penalties in nonconvex constrained optimization: diminishing stepsizes and iteration complexity
• scientific article; zbMATH DE number 7626711 (Why is no real title available?)
• Distributed Optimization Based on Gradient Tracking Revisited: Enhancing Convergence Rate via Surrogation
• Combining approximation and exact penalty in hierarchical programming
• Decentralized dictionary learning over time-varying digraphs
• Scalable subspace methods for derivative-free nonlinear least-squares optimization
• Localization and approximations for distributed non-convex optimization

This page was built for publication: Parallel Selective Algorithms for Nonconvex Big Data Optimization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4580494)