Optimal selection of sample-size dependent common subsets of covariates for multi-task regression prediction

DOI10.1214/21-EJS1901MaRDI QIDQ2074281zbMATH OpenOpenAlexFDO

Authors David Azriel, Yosef Rinott

Publication date 9 February 2022

Published in Electronic Journal of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2012.05949, https://projecteuclid.org/journals/electronic-journal-of-statistics/volume-15/issue-2/Optimal-selection-of-sample-size-dependent-common-subsets-of-covariates/10.1214/21-EJS1901.full

zbMATH Keywords

equivalent number of observations (ENO)GENOMallows \(C_p\)model selectionoverfittingrandom covariatestransfer learning

Mathematics Subject Classification ID

Linear inference, regression (62J99)

Abstract: An analyst is given a training set consisting of regression datasets $D_j$ of different sizes, which are distributed according to some $G_j$, $j=1,ldots,calJ$, where the distributions $G_j$ are assumed to form a random sample generated by some common source. In particular, the $D_j$'s have a common set of covariates and they are all labeled. The training set is used by the analyst for selection of subsets of covariates denoted by $P^{\ast }(n)$, whose role is described next. The multi-task problem we consider is as follows: given a number of random labeled datasets (which may be in the training set or not) $D_{J_k}$ of size $n_k$, $k=1,ldots,K$, estimate separately for each dataset the regression coefficients on the subset of covariates $P^{\ast }(n_k)$ and then predict future dependent variables given their covariates. Naturally, a large sample size $n_k$ of $D_{J_k}$ allows a larger subset of covariates, and the dependence of the size of the selected covariate subsets on $n_k$ is needed in order to achieve good prediction and avoid overfitting. Subset selection is notoriously difficult and computationally demanding, and requires large samples; using all the regression datasets in the training set together amounts to borrowing strength toward better selection under suitable assumptions. Furthermore, using common subsets for all regressions having a given sample size standardizes and simplifies the data collection and avoids having to select and use a different subset for each prediction task. Our approach is efficient when the relevant covariates for prediction are common to the different regressions, while the models' coefficients may vary between different regressions.

Recommendations

• Subset selection for multiple linear regression via optimization
• Consistent variable selection criteria in multivariate linear regression even when dimension exceeds sample size
• scientific article; zbMATH DE number 4176261
• scientific article; zbMATH DE number 3892369
• Optimal subsampling for multiplicative regression with massive data
• scientific article; zbMATH DE number 475222
• Variable selection for classification and regression in large p, small n problems
• Complete subset regressions with large-dimensional sets of predictors
• Variable selection for estimating the optimal treatment regimes in the presence of a large number of covariates

Cites work

• A new look at the statistical model identification
• A note on the expected value of an inverse matrix
• A Tale of Two Regressions
• Adaptive covariate acquisition for minimizing total cost of classification
• Asymptotic properties of criteria for selection of variables in multiple regression
• Bagging predictors
• Can the strengths of AIC and BIC be shared? A conflict between model indentification and regression estimation
• Estimating the dimension of a model
• From Fixed-X to Random-X Regression: Bias-Variance Decompositions, Covariance Penalties, and Prediction Error Estimation
• scientific article; zbMATH DE number 4084766 (Why is no real title available?)
• scientific article; zbMATH DE number 3782216 (Why is no real title available?)
• scientific article; zbMATH DE number 1034037 (Why is no real title available?)
• scientific article; zbMATH DE number 1550518 (Why is no real title available?)
• scientific article; zbMATH DE number 6125590 (Why is no real title available?)
• Identifying a minimal class of models for high-dimensional data
• Learning and equilibrium as useful approximations: accuracy of prediction on randomly selected constant sum games
• Maximum Likelihood Estimation of Misspecified Models
• Model assessment tools for a model false world
• Model Selection and Multimodel Inference
• Models as approximations. I. Consequences illustrated with linear regression
• Probability and stochastics.
• Some Comments on C P
• The Focused Information Criterion
• User-friendly tail bounds for sums of random matrices

This page was built for publication: Optimal selection of sample-size dependent common subsets of covariates for multi-task regression prediction

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2074281)