High-dimensional composite quantile regression: optimal statistical guarantees and fast algorithms

DOI10.1214/23-EJS2147arXiv2208.09817MaRDI QIDQ6184871FDOQ6184871

Authors:

Publication date: 5 January 2024

Published in: Electronic Journal of Statistics (Search for Journal in Brave)

Abstract: The composite quantile regression (CQR) was introduced by Zou and Yuan [Ann. Statist. 36 (2008) 1108--1126] as a robust regression method for linear models with heavy-tailed errors while achieving high efficiency. Its penalized counterpart for high-dimensional sparse models was recently studied in Gu and Zou [IEEE Trans. Inf. Theory 66 (2020) 7132--7154], along with a specialized optimization algorithm based on the alternating direct method of multipliers (ADMM). Compared to the various first-order algorithms for penalized least squares, ADMM-based algorithms are not well-adapted to large-scale problems. To overcome this computational hardness, in this paper we employ a convolution-smoothed technique to CQR, complemented with iteratively reweighted $el{l}_1$-regularization. The smoothed composite loss function is convex, twice continuously differentiable, and locally strong convex with high probability. We propose a gradient-based algorithm for penalized smoothed CQR via a variant of the majorize-minimization principal, which gains substantial computational efficiency over ADMM. Theoretically, we show that the iteratively reweighted $el{l}_1$-penalized smoothed CQR estimator achieves near-minimax optimal convergence rate under heavy-tailed errors without any moment constraint, and further achieves near-oracle convergence rate under a weaker minimum signal strength condition than needed in Gu and Zou (2020). Numerical studies demonstrate that the proposed method exhibits significant computational advantages without compromising statistical performance compared to two state-of-the-art methods that achieve robustness and high efficiency simultaneously.

Full work available at URL: https://arxiv.org/abs/2208.09817

Recommendations

• Composite quantile regression for ultra-high dimensional semiparametric model averaging
• Composite quantile regression for massive datasets
• Sparse Composite Quantile Regression with Ultra-high Dimensional Heterogeneous Data
• Single-index composite quantile regression for ultra-high-dimensional data
• Sparse Composite Quantile Regression in Ultrahigh Dimensions With Tuning Parameter Calibration
• Advanced algorithms for penalized quantile and composite quantile regression
• Optimal subsampling algorithms for composite quantile regression in massive data
• A note on the efficiency of composite quantile regression
• An effective method to reduce the computational complexity of composite quantile regression
• Distributed Sparse Composite Quantile Regression in Ultrahigh Dimensions

zbMATH Keywords

high-dimensional dataasymptotic efficiencysparsityoracle propertycomposite quantile regressionconvolution smoothing

Mathematics Subject Classification ID

Foundations and philosophical topics in statistics (62A01) Ridge regression; shrinkage estimators (Lasso) (62J07)

Cites Work

• Nearly unbiased variable selection under minimax concave penalty
• Weak convergence and empirical processes. With applications to statistics
• Regression Quantiles
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• Title not available (Why is that?)
• Statistics for high-dimensional data. Methods, theory and applications.
• One-step sparse estimates in nonconcave penalized likelihood models
• Simultaneous analysis of Lasso and Dantzig selector
• Atomic Decomposition by Basis Pursuit
• Title not available (Why is that?)
• \(\ell_1\)-penalized quantile regression in high-dimensional sparse models
• Title not available (Why is that?)
• High-dimensional statistics. A non-asymptotic viewpoint
• High-dimensional probability. An introduction with applications in data science
• Concentration inequalities. A nonasymptotic theory of independence
• Title not available (Why is that?)
• Statistical consistency and asymptotic normality for high-dimensional robust \(M\)-estimators
• Local Composite Quantile Regression Smoothing: An Efficient and Safe Alternative to Local Polynomial Regression
• Composite quantile regression and the oracle model selection theory
• Reconstruction From Anisotropic Random Measurements
• Estimation of High Dimensional Mean Regression in the Absence of Symmetry and Light Tail Assumptions
• A general theory of concave regularization for high-dimensional sparse estimation problems
• Optimization with sparsity-inducing penalties
• Model Selection via Bayesian Information Criterion for Quantile Regression Models
• Robust Variable Selection With Exponential Squared Loss
• Parametric estimation. Finite sample theory
• Minimum distance Lasso for robust high-dimensional regression
• Title not available (Why is that?)
• Penalized Composite Quasi-Likelihood for Ultrahigh Dimensional Variable Selection
• Parallelizing the dual revised simplex method
• Robust and consistent variable selection in high-dimensional generalized linear models
• Adaptive Huber Regression
• Smoothed quantile regression with large-scale inference
• I-LAMM for sparse learning: simultaneous control of algorithmic complexity and statistical error
• Statistical foundations of data science
• Scaling-up empirical risk minimization: optimization of incomplete \(U\)-statistics
• Rejoinder to “A Tuning-Free Robust and Efficient Approach to High-Dimensional Regression”
• Sparse Composite Quantile Regression in Ultrahigh Dimensions With Tuning Parameter Calibration
• High-Dimensional Quantile Regression: Convolution Smoothing and Concave Regularization

Cited In (6)

• Composite quantile regression for massive datasets
• An adaptive composite quantile approach to dimension reduction
• Sparse Composite Quantile Regression in Ultrahigh Dimensions With Tuning Parameter Calibration
• Distributed Sparse Composite Quantile Regression in Ultrahigh Dimensions
• Composite quantile regression for ultra-high dimensional semiparametric model averaging
• Advanced algorithms for penalized quantile and composite quantile regression

This page was built for publication: High-dimensional composite quantile regression: optimal statistical guarantees and fast algorithms

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6184871)