A new approach to variable selection in least squares problems

DOI10.1093/imanum/20.3.389zbMath0962.65036OpenAlexW2062102668MaRDI QIDQ4504955

Turlach, Berwin A., Presnell, Brett, Michael R. Osborne

Publication date: 30 January 2001

Published in: IMA Journal of Numerical Analysis (Search for Journal in Brave)

Full work available at URL: https://semanticscholar.org/paper/db31973871f5648cffb02e7590582376e16ad0bb

zbMATH Keywords

homotopy method Gram-Schmidt orthogonalization descent method least squares problems exploratory data analysis design matrix stepwise regression selection mechanisms minimal sum of squares of residuals selection of columns

Mathematics Subject Classification ID

Linear regression; mixed models (62J05) Numerical solutions to overdetermined systems, pseudoinverses (65F20) Orthogonalization in numerical linear algebra (65F25)

Related Items

Ridge regression and the Lasso: how do they do as finders of significant regressors and their multipliers?, On the uniqueness of solutions for the basis pursuit in the continuum, Global optimization for sparse solution of least squares problems, Robust functional coefficient selection for the single-index varying coefficients regression model, Hierarchical Total Variations and Doubly Penalized ANOVA Modeling for Multivariate Nonparametric Regression, A General Theory of Singular Values with Applications to Signal Denoising, Sparsely restricted penalized estimators, Monte Carlo Simulation for Lasso-Type Problems by Estimator Augmentation, Variable selection using penalised likelihoods for point patterns on a linear network, High-dimensional sparse index tracking based on a multi-step convex optimization approach, Distributed Sparse Composite Quantile Regression in Ultrahigh Dimensions, Matrix-wise $\ell_0$-constrained sparse nonnegative least squares, Screening Rules and its Complexity for Active Set Identification, Variable selection approach for zero-inflated count data via adaptive lasso, A fast homotopy algorithm for gridless sparse recovery, The homotopy method revisited: Computing solution paths of $\ell _1$-regularized problems, Unnamed Item, A fast unified algorithm for solving group-lasso penalize learning problems, A general theory of concave regularization for high-dimensional sparse estimation problems, Edge selection for undirected graphs, Imaging with highly incomplete and corrupted data, Generalized Sobol sensitivity indices for dependent variables: numerical methods, A Generic Path Algorithm for Regularized Statistical Estimation, Unnamed Item, A time continuation based fast approximate algorithm for compressed sensing related optimization, Unnamed Item, Mixed Lasso estimator for stochastic restricted regression models, Discussion: ``A significance test for the lasso, Discussion: ``A significance test for the lasso, Discussion: ``A significance test for the lasso, Discussion: ``A significance test for the lasso, Discussion: ``A significance test for the lasso, Solving Basis Pursuit, Geometry and Symmetry in Short-and-Sparse Deconvolution, Removal of mixed Gaussian and impulse noise using directional tensor product complex tight framelets, Least angle regression. (With discussion), Are financial ratios relevant for trading credit risk? Evidence from the CDS market, Accelerating gradient projection methods for $\ell _1$-constrained signal recovery by steplength selection rules, Forecasting economic time series using targeted predictors, Low-Rank and Sparse Dictionary Learning, Large sparse signal recovery by conjugate gradient algorithm based on smoothing technique, Robust Decoding from 1-Bit Compressive Sampling with Ordinary and Regularized Least Squares, Optimality conditions for the constrainedL_p-regularization, A primal-dual homotopy algorithm for $\ell _{1}$-minimization with $\ell _{\infty }$-constraints, Spectral Decompositions Using One-Homogeneous Functionals, Estimation for High-Dimensional Linear Mixed-Effects Models Using ℓ1-Penalization, Optimal Quadrature-Sparsification for Integral Operator Approximation, Nonlinear spectral analysis via one-homogeneous functionals: overview and future prospects, Simultaneous analysis of Lasso and Dantzig selector, Computational Spectral and Ultrafast Imaging via Convex Optimization, A weighted $\ell_1$-minimization approach for sparse polynomial chaos expansions, Natural coordinate descent algorithm for $\ell_1$-penalised regression in generalised linear models, Inversion of electromagnetic geosoundings using coordinate descent optimization, ParNes: A rapidly convergent algorithm for accurate recovery of sparse and approximately sparse signals, Isotropic sparse regularization for spherical harmonic representations of random fields on the sphere, The degrees of freedom of partly smooth regularizers, Forward and backward least angle regression for nonlinear system identification, Noise-robust semi-supervised learning via fast sparse coding, Model selection via standard error adjusted adaptive Lasso, Non-asymptotic oracle inequalities for the Lasso and Group Lasso in high dimensional logistic model, Sparse conjugate directions pursuit with application to fixed-size kernel models, A simple homotopy proximal mapping algorithm for compressive sensing, Adaptive Dantzig density estimation, Support union recovery in high-dimensional multivariate regression, On the structure of regularization paths for piecewise differentiable regularization terms, Construction, management, and performance of sparse Markowitz portfolios, Compressive Sensing, Inferring multiple graphical structures, The Lasso problem and uniqueness, Honest variable selection in linear and logistic regression models via $\ell _{1}$ and $\ell _{1}+\ell _{2}$ penalization, Learning and estimation applications of an online homotopy algorithm for a generalization of the LASSO, Simultaneous multiple response regression and inverse covariance matrix estimation via penalized Gaussian maximum likelihood, Boosting algorithms: regularization, prediction and model fitting, Optimization methods for regularization-based ill-posed problems: a survey and a multi-objective framework, Efficient LED-SAC sparse estimator using fast sequential adaptive coordinate-wise optimization (LED-2SAC), Bayesian projection approaches to variable selection in generalized linear models, An infeasible-point subgradient method using adaptive approximate projections, On solving $L_{q}$-penalized regressions, One-step sparse estimates in nonconcave penalized likelihood models, Discussion: One-step sparse estimates in nonconcave penalized likelihood models, The sparsity and bias of the LASSO selection in high-dimensional linear regression, Path following in the exact penalty method of convex programming, Level-set methods for convex optimization, Efficient histogram dictionary learning for text/image modeling and classification, Pathwise coordinate optimization, On the ``degrees of freedom of the lasso, An ordinary differential equation-based solution path algorithm, A significance test for the lasso, Discussion: ``A significance test for the lasso, Rejoinder: ``A significance test for the lasso, Projected gradient approach to the numerical solution of the SCoTLASS, Local behavior of sparse analysis regularization: applications to risk estimation, A Projection Proximal-Point Algorithm for ℓ¹Minimization, A coordinate gradient descent method for $\ell_{1}$-regularized convex minimization, Structured variable selection and estimation, An efficient algorithm for structured sparse quantile regression, A non-adapted sparse approximation of PDEs with stochastic inputs, Solution paths for the generalized Lasso with applications to spatially varying coefficients regression, Least angle and $\ell _{1}$ penalized regression: a review, Ways to sparse representation: An overview, L1Packv2: A Mathematica package for minimizing an $\ell _{1}$-penalized functional, Model-assisted inference for treatment effects using regularized calibrated estimation with high-dimensional data, Complexity and applications of the homotopy principle for uniformly constrained sparse minimization, Nearly unbiased variable selection under minimax concave penalty, Post-Pareto analysis and a new algorithm for the optimal parameter tuning of the elastic net, When do stepwise algorithms meet subset selection criteria?, Sorted concave penalized regression, Variable Selection for Semiparametric Varying Coefficient Partially Linear Errors-in-Variables (EV) Model with Missing Response, Low Complexity Regularization of Linear Inverse Problems, The stochastic properties of $\ell^1$-regularized spherical Gaussian fields, Sparse and stable Markowitz portfolios, SGL-SVM: a novel method for tumor classification via support vector machine with sparse group lasso, Variable Selection in Linear Mixed Models Using an Extended Class of Penalties, Linear convergence of iterative soft-thresholding, ROS regression: integrating regularization with optimal scaling regression, A modified local quadratic approximation algorithm for penalized optimization problems, A constrained optimization reformulation and a feasible descent direction method for $L_{1/2}$ regularization, Bayesian adaptive Lasso, APPLE: approximate path for penalized likelihood estimators, Majorization minimization by coordinate descent for concave penalized generalized linear models, Another look at linear programming for feature selection via methods of regularization, A coordinate descent homotopy method for linearly constrained nonsmooth convex minimization, The Group Lasso for Logistic Regression, Sharp oracle inequalities for low-complexity priors, Input selection and shrinkage in multiresponse linear regression, $h$-$p$ adaptive model based approximation of moment free sensitivity indices, Sampling from non-smooth distributions through Langevin diffusion, Smoothing Newton method for $\ell^0$-$\ell^2$ regularized linear inverse problem, Asymptotic linear expansion of regularized M-estimators, A review on restoration of seismic wavefields based on regularization and compressive sensing, The composite absolute penalties family for grouped and hierarchical variable selection, Conditional selective inference for robust regression and outlier detection using piecewise-linear homotopy continuation, An Overview of Computational Sparse Models and Their Applications in Artificial Intelligence, LASSO for streaming data with adaptative filtering