Detangling robustness in high dimensions: composite versus model-averaged estimation (Q2192312)

Consider a linear regression problem \(Y=X \beta + \epsilon\), with the design matrix \(X\), the vector \(\beta\) of unknown parameters to be estimated, and the vector \(Y\) of observations. The error vector \(\epsilon\) has independent, identically distributed, zero-mean components. Two types of weighted estimators are considered: i) model-averaged estimators, where estimators from different estimation methods or models are weighted and averaged, and ii) composite estimators, where weighted different loss functions are averaged. Moreover, regularized estimators and estimators with optimized weights are considered. The performance of the estimators is compared, where especially the uncertainty in the selection of the weights is taken into account. Computational results are given.