Transductive versions of the Lasso and the Dantzig selector

DOI10.1016/J.JSPI.2012.03.020MaRDI QIDQ447611zbMATH OpenOpenAlexFDO

Authors Pierre Alquier, Mohamed Hebiri

Publication date 4 September 2012

Published in Journal of Statistical Planning and Inference (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0906.0652

zbMATH Keywords

Lassovariable selectionhigh-dimensional datasparsityhigh-dimensional regression estimationtransduction

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Linear regression; mixed models (62J05) Ridge regression; shrinkage estimators (Lasso) (62J07)

Abstract: We consider the linear regression problem, where the number $p$ of covariates is possibly larger than the number $n$ of observations $(x_i,y_i{)}_{ileqileqn}$, under sparsity assumptions. On the one hand, several methods have been successfully proposed to perform this task, for example the LASSO or the Dantzig Selector. On the other hand, consider new values $(x_i{)}_{n+1leqileqm}$. If one wants to estimate the corresponding $y_i$'s, one should think of a specific estimator devoted to this task, referred by Vapnik as a "transductive" estimator. This estimator may differ from an estimator designed to the more general task "estimate on the whole domain". In this work, we propose a generalized version both of the LASSO and the Dantzig Selector, based on the geometrical remarks about the LASSO in pr'evious works. The "usual" LASSO and Dantzig Selector, as well as new estimators interpreted as transductive versions of the LASSO, appear as special cases. These estimators are interesting at least from a theoretical point of view: we can give theoretical guarantees for these estimators under hypotheses that are relaxed versions of the hypotheses required in the papers about the "usual" LASSO. These estimators can also be efficiently computed, with results comparable to the ones of the LASSO.

Recommendations

• On the prediction loss of the Lasso in the partially labeled setting
• Simultaneous analysis of Lasso and Dantzig selector
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• scientific article; zbMATH DE number 2243369
• Persistene in high-dimensional linear predictor-selection and the virtue of overparametrization

Cites work

• scientific article; zbMATH DE number 5957307 (Why is no real title available?)
• scientific article; zbMATH DE number 5957408 (Why is no real title available?)
• scientific article; zbMATH DE number 5544465 (Why is no real title available?)
• scientific article; zbMATH DE number 1332320 (Why is no real title available?)
• scientific article; zbMATH DE number 3444596 (Why is no real title available?)
• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• scientific article; zbMATH DE number 1391397 (Why is no real title available?)
• Aggregation by Exponential Weighting and Sharp Oracle Inequalities
• Aggregation for Gaussian regression
• Consistency of the group Lasso and multiple kernel learning
• DASSO: Connections Between the Dantzig Selector and Lasso
• Estimating the dimension of a model
• Generalization of constraints for high dimensional regression problems
• High-dimensional additive modeling
• High-dimensional generalized linear models and the lasso
• High-dimensional graphs and variable selection with the Lasso
• Lasso, iterative feature selection and the correlation selector: oracle inequalities and numerical performances
• Lasso-type recovery of sparse representations for high-dimensional data
• Least angle regression. (With discussion)
• On the Non-Negative Garrotte Estimator
• On the conditions used to prove oracle results for the Lasso
• On transductive support vector machines
• Pathwise coordinate optimization
• Simultaneous analysis of Lasso and Dantzig selector
• Smoothing \(\ell_1\)-penalized estimators for high-dimensional time-course data
• Some theoretical results on the grouped variables Lasso
• Sparse recovery in convex hulls via entropy penalization
• Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators
• Text classification from labeled and unlabeled documents using EM
• The Adaptive Lasso and Its Oracle Properties
• The Dantzig selector and sparsity oracle inequalities
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties

Cited in

(3)

• On the prediction loss of the Lasso in the partially labeled setting
• DASSO: Connections Between the Dantzig Selector and Lasso
• Optimal and Safe Estimation for High-Dimensional Semi-Supervised Learning

Describes a project that uses

Uses Software

• hgam

This page was built for publication: Transductive versions of the Lasso and the Dantzig selector

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q447611)