SLOPE-adaptive variable selection via convex optimization

DOI10.1214/15-AOAS842zbMATH Open1454.62212arXiv1407.3824OpenAlexW1916786071WikidataQ40166307 ScholiaQ40166307MaRDI QIDQ902886FDOQ902886

Chiara Sabatti, Ewout van den Berg, Małgorzata Bogdan, Małgorzata Bogdan, Chiara Sabatti, Weijie Su, Emmanuel J. Candès, Weijie Su, Ewout van Den Berg, Emmanuel J. Candès

Publication date: 4 January 2016

Published in: The Annals of Applied Statistics (Search for Journal in Brave)

Abstract: We introduce a new estimator for the vector of coefficients $$ in the linear model $$, where $X$ has dimensions $nimesp$ with $p$ possibly larger than $n$. SLOPE, short for Sorted L-One Penalized Estimation, is the solution to [min_{binmathbb{R}^p}frac{1}{2}Vert y-XbVert _{ell_2}^2+lambda_1vert bvert _{(1)}+lambda_2vert bvert_{(2)}+cdots+lambda_pvert bvert_{(p)},] where $lambd{a}_{1}gelambd{a}_{2}gecdotsgelambd{a}_{p}ge0$ and $vertbver{t}_{(1)}gevertbver{t}_{(2)}gecdotsgevertbver{t}_{(p)}$ are the decreasing absolute values of the entries of $b$. This is a convex program and we demonstrate a solution algorithm whose computational complexity is roughly comparable to that of classical $el{l}_1$ procedures such as the Lasso. Here, the regularizer is a sorted $el{l}_1$ norm, which penalizes the regression coefficients according to their rank: the higher the rank - that is, stronger the signal - the larger the penalty. This is similar to the Benjamini and Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289-300] procedure (BH) which compares more significant $p$-values with more stringent thresholds. One notable choice of the sequence $lambd{a}_i$ is given by the BH critical values $lambd{a}_{mathrmBH}(i)=z(1-icdotq/2p)$, where $qin(0,1)$ and $z(alpha)$ is the quantile of a standard normal distribution. SLOPE aims to provide finite sample guarantees on the selected model; of special interest is the false discovery rate (FDR), defined as the expected proportion of irrelevant regressors among all selected predictors. Under orthogonal designs, SLOPE with $lambd{a}_{mathrmBH}$ provably controls FDR at level $q$. Moreover, it also appears to have appreciable inferential properties under more general designs $X$ while having substantial power, as demonstrated in a series of experiments running on both simulated and real data.

Full work available at URL: https://arxiv.org/abs/1407.3824

zbMATH Keywords

Lassovariable selectionfalse discovery ratesparse regressionsorted \(\ell_{1}\) penalized estimation (SLOPE)

Mathematics Subject Classification ID

Linear regression; mixed models (62J05) Ridge regression; shrinkage estimators (Lasso) (62J07)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
• Templates for convex cone problems with applications to sparse signal recovery
• The Adaptive Lasso and Its Oracle Properties
• Some Comments on C P
• A new look at the statistical model identification
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
• p-Values for High-Dimensional Regression
• Enhancing sparsity by reweighted \(\ell _{1}\) minimization
• On asymptotically optimal confidence regions and tests for high-dimensional models
• Introductory lectures on convex optimization. A basic course.
• Minimax detection of a signal for \(l^ n\)-balls.
• A significance test for the lasso
• Adapting to unknown sparsity by controlling the false discovery rate
• Hypothesis Testing in High-Dimensional Regression Under the Gaussian Random Design Model: Asymptotic Theory
• Scaled sparse linear regression
• Valid post-selection inference
• Statistical significance in high-dimensional linear models
• Gaussian model selection
• \(\ell_{1}\)-penalization for mixture regression models
• High-dimensional variable selection
• Nonmetric multidimensional scaling. A numerical method
• Relaxed Lasso
• The risk inflation criterion for multiple regression
• Controlling the false discovery rate via knockoffs
• Projections onto order simplexes
• Some results on false discovery rate in stepwise multiple testing procedures.
• Active set algorithms for isotonic regression; a unifying framework
• SLOPE-adaptive variable selection via convex optimization
• Tweedie’s Formula and Selection Bias
• Asymptotic Bayes-optimality under sparsity of some multiple testing procedures
• Model selection and sharp asymptotic minimaxity
• The Covariance Inflation Criterion for Adaptive Model Selection
• Local asymptotic coding and the minimum description length
• False Discovery Rate–Adjusted Multiple Confidence Intervals for Selected Parameters
• Model selection by multiple test procedures
• A simple forward selection procedure based on false discovery rate control
• Some optimality properties of FDR controlling rules under sparsity

Cited In (83)

• SLOPE-adaptive variable selection via convex optimization
• High-dimensional inference: confidence intervals, \(p\)-values and R-software \texttt{hdi}
• Optimal false discovery control of minimax estimators
• False Discovery Rate Control via Data Splitting
• Estimating minimum effect with outlier selection
• A consistent and numerically efficient variable selection method for sparse Poisson regression with applications to learning and signal recovery
• Adapting Regularized Low-Rank Models for Parallel Architectures
• Overcoming the limitations of phase transition by higher order analysis of regularization techniques
• On spike and slab empirical Bayes multiple testing
• Sorted concave penalized regression
• Title not available (Why is that?)
• Facilitating OWL norm minimizations
• A Unifying Tutorial on Approximate Message Passing
• Familywise error rate control via knockoffs
• Canonical thresholding for nonsparse high-dimensional linear regression
• Solving the OSCAR and SLOPE Models Using a Semismooth Newton-Based Augmented Lagrangian Method
• Regularization and the small-ball method. I: Sparse recovery
• Improved bounds for square-root Lasso and square-root slope
• Efficient projection algorithms onto the weighted \(\ell_1\) ball
• Oracle inequalities for high-dimensional prediction
• Slope meets Lasso: improved oracle bounds and optimality
• Online rules for control of false discovery rate and false discovery exceedance
• SLOPE is adaptive to unknown sparsity and asymptotically minimax
• Sharp oracle inequalities for low-complexity priors
• Estimation bounds and sharp oracle inequalities of regularized procedures with Lipschitz loss functions
• Robust machine learning by median-of-means: theory and practice
• Empirical Bayes cumulative \(\ell\)-value multiple testing procedure for sparse sequences
• Detecting multiple replicating signals using adaptive filtering procedures
• A Critical Review of LASSO and Its Derivatives for Variable Selection Under Dependence Among Covariates
• Proximal operator for the sorted \(\ell_1\) norm: application to testing procedures based on SLOPE
• On the asymptotic properties of SLOPE
• A flexible shrinkage operator for fussy grouped variable selection
• Model selection with mixed variables on the Lasso path
• Iteratively reweighted \(\ell_1\)-penalized robust regression
• Adaptive Huber Regression
• Sharp Oracle Inequalities for Square Root Regularization
• Model Selection With Lasso-Zero: Adding Straw to the Haystack to Better Find Needles
• Variable selection consistency of Gaussian process regression
• Predictor ranking and false discovery proportion control in high-dimensional regression
• Variable selection via adaptive false negative control in linear regression
• On the exponentially weighted aggregate with the Laplace prior
• Variable selection with Hamming loss
• Simple expressions of the LASSO and SLOPE estimators in low-dimension
• Feasibility and a fast algorithm for Euclidean distance matrix optimization with ordinal constraints
• Degrees of freedom in submodular regularization: a computational perspective of Stein's unbiased risk estimate
• On the sparsity of Mallows model averaging estimator
• Randomized Gradient Boosting Machine
• The Trimmed Lasso: Sparse Recovery Guarantees and Practical Optimization by the Generalized Soft-Min Penalty
• False Discovery Rate Control Under General Dependence By Symmetrized Data Aggregation
• Regularization and the small-ball method II: complexity dependent error rates
• Iterative gradient descent for outlier detection
• Title not available (Why is that?)
• Learning from MOM's principles: Le Cam's approach
• Statistical proof? The problem of irreproducibility
• Fundamental barriers to high-dimensional regression with convex penalties
• Iterative algorithm for discrete structure recovery
• Sparse optimization via vector \(k\)-norm and DC programming with an application to feature selection for support vector machines
• A power analysis for Model-X knockoffs with \(\ell_p\)-regularized statistics
• An easily implementable algorithm for efficient projection onto the ordered weighted \(\ell_1\) norm ball
• Adaptive novelty detection with false discovery rate guarantee
• Robust and tuning-free sparse linear regression via square-root slope
• Dual formulation of the sparsity constrained optimization problem: application to classification
• Pattern recovery and signal denoising by SLOPE when the design matrix is orthogonal
• Low-rank tensor regression for selection of grouped variables
• Characterizing the SLOPE trade-off: a variational perspective and the Donoho-Tanner limit
• Independently Interpretable Lasso for Generalized Linear Models
• On Lasso and Slope drift estimators for Lévy-driven Ornstein-Uhlenbeck processes
• Safe Rules for the Identification of Zeros in the Solutions of the SLOPE Problem
• Approximate Selective Inference via Maximum Likelihood
• Optimized variable selection via repeated data splitting
• A tradeoff between false discovery and true positive proportions for sparse high-dimensional logistic regression
• Comment: Feature Screening and Variable Selection via Iterative Ridge Regression
• Nonconvex Dantzig selector and its parallel computing algorithm
• Variable Selection With Second-Generation P-Values
• Efficient path algorithms for clustered Lasso and OSCAR
• Sparse index clones via the sorted ℓ₁-Norm
• Group SLOPE – Adaptive Selection of Groups of Predictors
• geneSLOPE
• Nested model averaging on solution path for high-dimensional linear regression
• Hedonic pricing modelling with unstructured predictors: an application to Italian fashion industry
• Classification in high dimension with an Alternative Feature Augmentation via Nonparametrics and Selection (AFANS)
• Rates of convergence of the adaptive elastic net and the post-selection procedure in ultra-high dimensional sparse models
• Adaptive Bayesian SLOPE: Model Selection With Incomplete Data

Uses Software

• OSCAR
• TFOCS
• isotone
• covTest

This page was built for publication: SLOPE-adaptive variable selection via convex optimization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q902886)