Nonpenalized variable selection in high-dimensional linear model settings via generalized fiducial inference

DOI10.1214/18-AOS1733MaRDI QIDQ2414104zbMATH OpenWikidataFDO

Authors Jonathan P. Williams, Jan Hannig

Publication date 10 May 2019

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1702.07283, https://projecteuclid.org/euclid.aos/1550026855

zbMATH Keywords

\(L_{0}\) minimizationbest subset selectionfeature selectionhigh-dimensional regression

Mathematics Subject Classification ID

Foundations and philosophical topics in statistics (62A01) Asymptotic properties of parametric estimators (62F12) Linear regression; mixed models (62J05)

Abstract: Standard penalized methods of variable selection and parameter estimation rely on the magnitude of coefficient estimates to decide which variables to include in the final model. However, coefficient estimates are unreliable when the design matrix is collinear. To overcome this challenge an entirely new perspective on variable selection is presented within a generalized fiducial inference framework. This new procedure is able to effectively account for linear dependencies among subsets of covariates in a high-dimensional setting where $p$ can grow almost exponentially in $n$, as well as in the classical setting where $plen$. It is shown that the procedure very naturally assigns small probabilities to subsets of covariates which include redundancies by way of explicit $L_0$ minimization. Furthermore, with a typical sparsity assumption, it is shown that the proposed method is consistent in the sense that the probability of the true sparse subset of covariates converges in probability to 1 as $noinfty$, or as $noinfty$ and $poinfty$. Very reasonable conditions are needed, and little restriction is placed on the class of possible subsets of covariates to achieve this consistency result.

Review text

This article develops a new perspective on variable selection to deal with linear dependences among subsets of covariates in a high-dimensional setting. A generalized fiducial inference framework is adopted. A procedure to assign small probabilities to subsets of covariates which include redundancies by way of explicit Lo minimization using a Dantzig selector approach is presented. A concept of $\varepsilon$-admissible subsets is proposed to give rise to the determination of a posterior-like distribution which assigns negligible probabilities to all subsets with redundancies in the true data generating model. It is shown that, under a typical sparsity condition, the probability of the true data generating model converges to 1. Comparisons with Bayesian and frequentist methods are performed in two simulation setups.

Recommendations

• Generalized fiducial inference for ultrahigh-dimensional regression
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• scientific article; zbMATH DE number 7108807
• Variable selection using stepdown procedures in high-dimensional linear models
• Variable selection in high-dimensional linear models: partially faithful distributions and the PC-simple algorithm

Cited in

(13)

• Generalized fiducial factor: an alternative to the Bayes factor for forensic identification of source problems
• Generalized fiducial inference for ultrahigh-dimensional regression
• Covariance estimation via fiducial inference
• Generalized fiducial methods for testing quantitative trait locus effects in genetic backcross studies
• Discussion of “A Gibbs Sampler for a Class of Random Convex Polytopes”
• The EAS approach for graphical selection consistency in vector autoregression models
• Generalized fiducial methods for testing the homogeneity of a three-sample problem with a mixture structure
• The EAS approach to variable selection for multivariate response data in high-dimensional settings
• Simultaneous confidence intervals for mean differences of multiple zero-inflated gamma distributions with applications to precipitation
• The quasi-fiducial model selection for Kriging model
• Demystifying inferential models and confidence curves: a fiducial perspective
• Possibilistic Inferential Models: A Review
• A Bayesian shrinkage estimator for transfer learning

This page was built for publication: Nonpenalized variable selection in high-dimensional linear model settings via generalized fiducial inference

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2414104)