Optimal selection of sample-size dependent common subsets of covariates for multi-task regression prediction

DOI10.1214/21-EJS1901zbMATH Open1493.62456arXiv2012.05949OpenAlexW3196668636MaRDI QIDQ2074281FDOQ2074281

Publication date: 9 February 2022

Published in: Electronic Journal of Statistics (Search for Journal in Brave)

Abstract: An analyst is given a training set consisting of regression datasets

D_{j}

of different sizes, which are distributed according to some

G_{j}

,

j = 1, l d o t s, c a l J

, where the distributions

G_{j}

are assumed to form a random sample generated by some common source. In particular, the

D_{j}

's have a common set of covariates and they are all labeled. The training set is used by the analyst for selection of subsets of covariates denoted by

P^{*} (n)

, whose role is described next. The multi-task problem we consider is as follows: given a number of random labeled datasets (which may be in the training set or not)

D_{J_{k}}

of size

n_{k}

,

k = 1, l d o t s, K

, estimate separately for each dataset the regression coefficients on the subset of covariates

P^{*} (n_{k})

and then predict future dependent variables given their covariates. Naturally, a large sample size

n_{k}

of

D_{J_{k}}

allows a larger subset of covariates, and the dependence of the size of the selected covariate subsets on

n_{k}

is needed in order to achieve good prediction and avoid overfitting. Subset selection is notoriously difficult and computationally demanding, and requires large samples; using all the regression datasets in the training set together amounts to borrowing strength toward better selection under suitable assumptions. Furthermore, using common subsets for all regressions having a given sample size standardizes and simplifies the data collection and avoids having to select and use a different subset for each prediction task. Our approach is efficient when the relevant covariates for prediction are common to the different regressions, while the models' coefficients may vary between different regressions.

Full work available at URL: https://arxiv.org/abs/2012.05949

zbMATH Keywords

model selection transfer learning overfitting random covariates equivalent number of observations (ENO)GENO Mallows \(C_p\)

Mathematics Subject Classification ID

Linear inference, regression (62J99)

Cites Work

Recommendations

This page was built for publication: Optimal selection of sample-size dependent common subsets of covariates for multi-task regression prediction

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2074281)