Sign-constrained least squares estimation for high-dimensional regression
From MaRDI portal
Abstract: Many regularization schemes for high-dimensional regression have been put forward. Most require the choice of a tuning parameter, using model selection criteria or cross-validation schemes. We show that a simple non-negative or sign-constrained least squares is a very simple and effective regularization technique for a certain class of high-dimensional regression problems. The sign constraint has to be derived via prior knowledge or an initial estimator but no further tuning or cross-validation is necessary. The success depends on conditions that are easy to check in practice. A sufficient condition for our results is that most variables with the same sign constraint are positively correlated. For a sparse optimal predictor, a non-asymptotic bound on the L1-error of the regression coefficients is then proven. Without using any further regularization, the regression vector can be estimated consistently as long as log(p) s/n -> 0 for n -> infty, where s is the sparsity of the optimal regression vector, p the number of variables and n sample size. Network tomography is shown to be an application where the necessary conditions for success of non-negative least squares are naturally fulfilled and empirical results confirm the effectiveness of the sign constraint for sparse recovery.
Recommendations
- The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
- Non-negative least squares for high-dimensional linear models: consistency and sparse recovery without regularization
- Square-root lasso: pivotal recovery of sparse signals via conic programming
- On constrained and regularized high-dimensional regression
- Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators
Cites work
- Aggregation for Gaussian regression
- An interior point Newton-like method for non-negative least-squares problems with degenerate solution
- Better Subset Regression Using the Nonnegative Garrote
- Estimating Network Loss Rates Using Active Tomography
- High-dimensional generalized linear models and the lasso
- scientific article; zbMATH DE number 409717 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- scientific article; zbMATH DE number 852525 (Why is no real title available?)
- Lasso-type recovery of sparse representations for high-dimensional data
- Learning the parts of objects by non-negative matrix factorization
- Least angle regression. (With discussion)
- Model Selection and Estimation in Regression with Grouped Variables
- Network Delay Tomography using Flexicast Experiments
- Network tomography: recent developments
- Non-negative least squares for high-dimensional linear models: consistency and sparse recovery without regularization
- Nonnegativity constraints in numerical analysis
- On the conditions used to prove oracle results for the Lasso
- On the Uniqueness of Nonnegative Sparse Solutions to Underdetermined Systems of Equations
- Random projections for the nonnegative least-squares problem
- Relaxed Lasso
- Ridge Regression: Biased Estimation for Nonorthogonal Problems
- Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$-Constrained Quadratic Programming (Lasso)
- Simultaneous analysis of Lasso and Dantzig selector
- Sparsity oracle inequalities for the Lasso
- Square-root lasso: pivotal recovery of sparse signals via conic programming
- The Adaptive Lasso and Its Oracle Properties
- The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
- The Group Square-Root Lasso: Theoretical Properties and Fast Algorithms
- The sparsity and bias of the LASSO selection in high-dimensional linear regression
Cited in
(31)- Group subset selection for linear regression
- Iteratively reweighted adaptive Lasso for conditional heteroscedastic time series with applications to AR-ARCH type processes
- High-dimensional sign-constrained feature selection and grouping
- Nonnegative estimation and variable selection under minimax concave penalty for sparse high-dimensional linear regression models
- Penalized and constrained LAD estimation in fixed and high dimension
- Bayesian inference for generalized linear model with linear inequality constraints
- Estimation of positive definite M-matrices and structure learning for attractive Gaussian Markov random fields
- Nonnegative adaptive Lasso for ultra-high dimensional regression models and a two-stage method applied in financial modeling
- Constrained inference in linear regression
- A component Lasso
- The geometry of least squares in the 21st century
- Non-negative least squares for high-dimensional linear models: consistency and sparse recovery without regularization
- Integer constraints for enhancing interpretability in linear regression
- Adaptive ridge estimator in a linear regression model with spherically symmetric error under constraint
- Nonnegative estimation and variable selection via adaptive elastic-net for high-dimensional data
- Efficient sparse portfolios based on composite quantile regression for high-dimensional index tracking
- High-dimensional least-squares with perfect positive correlation
- Sparse Topic Modeling: Computational Efficiency, Near-Optimal Algorithms, and Statistical Inference
- ESTIMATION OF A HIGH-DIMENSIONAL COUNTING PROCESS WITHOUT PENALTY FOR HIGH-FREQUENCY EVENTS
- Network Estimation by Mixing: Adaptivity and More
- Inequality restricted estimator for gamma regression: Bayesian approach as a solution to the multicollinearity
- The geometry of hypothesis testing over convex cones: generalized likelihood ratio tests and minimax radii
- Non-negative sparse recovery at minimal sampling rate
- An adaptive mixture view of particle filters
- Simplex constrained sparse optimization via tail screening
- Bayesian analysis of the beta regression model subject to linear inequality restrictions with application
- A non-asymptotic analysis of the single component PLS regression
- Vaccine design via nonnegative Lasso-based variable selection
- Bayesian quantile regression model with linear inequality constraints
- Bayesian modal regression with linear inequality constraints using mixture distributions
- On asymptotically optimal confidence regions and tests for high-dimensional models
This page was built for publication: Sign-constrained least squares estimation for high-dimensional regression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1954143)
