Converting ADMM to a proximal gradient for efficient sparse estimation

DOI10.1007/S42081-022-00150-6zbMATH Open1499.65228arXiv2104.10911OpenAlexW3215005297MaRDI QIDQ2103289FDOQ2103289

Ryosuke Shimmura, Joe Suzuki

Publication date: 13 December 2022

Published in: Japanese Journal of Statistics and Data Science (Search for Journal in Brave)

Abstract: In sparse estimation, such as fused lasso and convex clustering, we apply either the proximal gradient method or the alternating direction method of multipliers (ADMM) to solve the problem. It takes time to include matrix division in the former case, while an efficient method such as FISTA (fast iterative shrinkage-thresholding algorithm) has been developed in the latter case. This paper proposes a general method for converting the ADMM solution to the proximal gradient method, assuming that assumption that the derivative of the objective function is Lipschitz continuous. Then, we apply it to sparse estimation problems, such as sparse convex clustering and trend filtering, and we show by numerical experiments that we can obtain a significant improvement in terms of efficiency.

Full work available at URL: https://arxiv.org/abs/2104.10911

zbMATH Keywords

Lassoconvex optimizationLipschitz constantFISTA

Mathematics Subject Classification ID

Numerical mathematical programming methods (65K05) Convex programming (90C25) Ridge regression; shrinkage estimators (Lasso) (62J07)

Cites Work

• A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
• Estimating the dimension of a model
• Title not available (Why is that?)
• A new look at the statistical model identification
• Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers
• A dual algorithm for the solution of nonlinear variational problems via finite element approximation
• Sparsity and Smoothness Via the Fused Lasso
• Model Selection and Estimation in Regression with Grouped Variables
• On the global and linear convergence of the generalized alternating direction method of multipliers
• Title not available (Why is that?)
• Convex Analysis
• $\ell_1$ Trend Filtering
• Applications of a Splitting Algorithm to Decomposition in Convex Programming and Variational Inequalities
• Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming
• Proximité et dualité dans un espace hilbertien
• Alternating proximal gradient method for convex minimization
• A three-operator splitting scheme and its optimization applications
• Sparse Convex Clustering
• Sparse Estimation with Math and R

Cited In (3)

• Multi-task learning regression via convex clustering
• An incremental aggregated proximal ADMM for linearly constrained nonconvex optimization with application to sparse logistic regression problems
• ADMM Algorithmic Regularization Paths for Sparse Statistical Machine Learning

   Recommendations

• Fast algorithms for sparse inverse covariance estimation 👍 👎
• Linear convergence of the alternating direction method of multipliers for a class of convex optimization problems 👍 👎
• Title not available (Why is that?) 👍 👎
• On the linear convergence of the alternating direction method of multipliers 👍 👎
• Linearized alternating direction method of multipliers for sparse group and fused Lasso models 👍 👎

This page was built for publication: Converting ADMM to a proximal gradient for efficient sparse estimation

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2103289)