Quantile universal threshold

DOI10.1214/17-EJS1366MaRDI QIDQ131212zbMATH OpenOpenAlexFDO

Authors Caroline Giacobino, S. Sardy, Jairo Diaz-Rodriguez, Nick Hengartner, Sylvain Sardy

Publication date 1 January 2017

Published in Electronic Journal of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1511.05433, https://projecteuclid.org/euclid.ejs/1511492459

zbMATH Keywords

convex optimizationhigh-dimensionalityregularizationsparsitythresholding

Mathematics Subject Classification ID

Estimation in multivariate analysis (62H12) Ridge regression; shrinkage estimators (Lasso) (62J07) Learning and adaptive systems in artificial intelligence (68T05)

Abstract: Efficient recovery of a low-dimensional structure from high-dimensional data has been pursued in various settings including wavelet denoising, generalized linear models and low-rank matrix estimation. By thresholding some parameters to zero, estimators such as lasso, elastic net and subset selection allow to perform not only parameter estimation but also variable selection, leading to sparsity. Yet one crucial step challenges all these estimators: the choice of the threshold parameter~$lambda$. If too large, important features are missing; if too small, incorrect features are included. Within a unified framework, we propose a new selection of $lambda$ at the detection edge under the null model. To that aim, we introduce the concept of a zero-thresholding function and a null-thresholding statistic, that we explicitly derive for a large class of estimators. The new approach has the great advantage of transforming the selection of $lambda$ from an unknown scale to a probabilistic scale with the simple selection of a probability level. Numerical results show the effectiveness of our approach in terms of model selection and prediction.

Review text

The well known ``bias-variance trade-off dilemma appearing in machine learning theory is addressed. It is know that this dilemma can be handled by various regularization techniques starting from classical [\textit{A. N. Tikhonov}, Sov. Math., Dokl. 5, 1035--1038 (1963; Zbl 0141.11001); translation from Dokl. Akad. Nauk SSSR 151, 501--504 (1963); \textit{A. N. Tikhonov} and \textit{V. Y. Arsenin}, Solutions of ill-posed problems. New York etc.: John Wiley \& Sons; Washington, D.C.: V. H. Winston \& Sons (1977; Zbl 0354.65028)], followed by their Bayesian interpretation and many others. In this paper, the authors propose their approach using the novel concept of a zero-thresholding function and a null-thresholding statistic, that can be explicitly derived for a large class of estimators. The efficiency of the approach, called the quantile universal threshold, is demonstrated on synthetic and real data and implemented as \texttt{R} package \texttt{qut} which is available from the Comprehensive R Archive Network (CRAN).

Recommendations

• Thresholding in learning theory
• Regularized Bayesian estimation of generalized threshold regression models
• Bayesian regularized quantile regression
• Model selection in high-dimensional quantile regression with seamless L₀ penalty
• Best choices for regularization parameters in learning theory: on the bias-variance problem.

Cited in

(14)

• Random threshold for linear model selection, revisited
• A phase transition for finding needles in nonlinear haystacks with LASSO artificial neural networks
• Thresholding tests based on affine Lasso to achieve non-asymptotic nominal level and high power under sparse and dense alternatives in high dimension
• Low-rank model with covariates for count data with missing values
• qut
• Variable Selection With Second-Generation P-Values
• Model Selection With Lasso-Zero: Adding Straw to the Haystack to Better Find Needles
• _1-penalised ordinal polytomous regression estimators with application to gene expression studies
• Random thresholds for linear model selection
• A multi-resolution theory for approximating infinite-\(p\)-zero-\(n\): transitional inference, individualized predictions, and a world without bias-variance tradeoff
• Nonparametric Estimation of Galaxy Cluster Emissivity and Detection of Point Sources in Astrophysics With Two Lasso Penalties
• Sparse additive models in high dimensions with wavelets
• On the use of cross-validation for the calibration of the adaptive Lasso
• Tomographic reconstruction of disease transmission landscapes from GPS-recorded random paths

Describes a project that uses

Uses Software

• CRAN
• R
• qut

This page was built for publication: Quantile universal threshold

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q131212)