A global two-stage algorithm for non-convex penalized high-dimensional linear regression problems

DOI10.1007/S00180-022-01249-WMaRDI QIDQ6177008zbMATH OpenOpenAlexFDO

Authors Peili Li, Min Liu, Zhou Yu

Publication date 29 August 2023

Published in Computational Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2111.11801

zbMATH Keywords

difference of convex functionsglobal convergencehigh-dimensional linear regressionprimal dual active set with continuation algorithmtwo-stage algorithm

Mathematics Subject Classification ID

Computational methods for problems pertaining to statistics (62-08)

Abstract: By the asymptotic oracle property, non-convex penalties represented by minimax concave penalty (MCP) and smoothly clipped absolute deviation (SCAD) have attracted much attentions in high-dimensional data analysis, and have been widely used in signal processing, image restoration, matrix estimation, etc. However, in view of their non-convex and non-smooth characteristics, they are computationally challenging. Almost all existing algorithms converge locally, and the proper selection of initial values is crucial. Therefore, in actual operation, they often combine a warm-starting technique to meet the rigid requirement that the initial value must be sufficiently close to the optimal solution of the corresponding problem. In this paper, based on the DC (difference of convex functions) property of MCP and SCAD penalties, we aim to design a global two-stage algorithm for the high-dimensional least squares linear regression problems. A key idea for making the proposed algorithm to be efficient is to use the primal dual active set with continuation (PDASC) method, which is equivalent to the semi-smooth Newton (SSN) method, to solve the corresponding sub-problems. Theoretically, we not only prove the global convergence of the proposed algorithm, but also verify that the generated iterative sequence converges to a d-stationary point. In terms of computational performance, the abundant research of simulation and real data show that the algorithm in this paper is superior to the latest SSN method and the classic coordinate descent (CD) algorithm for solving non-convex penalized high-dimensional linear regression problems.

Cites work

• A highly efficient semismooth Newton augmented Lagrangian method for solving lasso problems
• A Primal Dual Active Set Algorithm With Continuation for Compressed Sensing
• A Statistical View of Some Chemometrics Regression Tools
• A unified primal dual active set algorithm for nonconvex sparse recovery
• Adapting to Unknown Smoothness via Wavelet Shrinkage
• Analysis of multi-stage convex relaxation for sparse regularization
• Atomic decomposition by basis pursuit
• Calculus of the exponent of Kurdyka-Łojasiewicz inequality and its applications to linear convergence of first-order methods
• Calibrating nonconvex penalized regression in ultra-high dimension
• Composite difference-MAX programs for modern statistical estimation problems
• Computing B-stationary points of nonsmooth DC programs
• Coordinate descent algorithms for lasso penalized regression
• Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection
• DC approximation approaches for sparse optimization
• Decoding by Linear Programming
• Difference-of-convex learning: directional stationarity, optimality, and sparsity
• Distributed optimization and statistical learning via the alternating direction method of multipliers
• High-dimensional graphs and variable selection with the Lasso
• scientific article; zbMATH DE number 5957408 (Why is no real title available?)
• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• scientific article; zbMATH DE number 7306909 (Why is no real title available?)
• Least angle regression. (With discussion)
• Nearly unbiased variable selection under minimax concave penalty
• One-step sparse estimates in nonconcave penalized likelihood models
• Proximal Newton-type methods for minimizing composite functions
• Proximity algorithms for image models: denoising
• Sequential Lasso cum EBIC for feature selection with ultra-high dimensional feature space
• Sparse Approximate Solutions to Linear Systems
• SparseNet: coordinate descent with nonconvex penalties
• The Convergence Guarantees of a Non-Convex Approach for Sparse Recovery
• The Primal-Dual Active Set Strategy as a Semismooth Newton Method
• Variable selection using MM algorithms
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties

Cited in

(2)

• A fast and effective algorithm for sparse linear regression with \(\ell_p\)-norm data fidelity and elastic net regularization
• Semi-proximal ADMM for fused Lasso penalized least absolute deviation in partially linear model

This page was built for publication: A global two-stage algorithm for non-convex penalized high-dimensional linear regression problems

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6177008)