Bootstrap tuning in Gaussian ordered model selection

DOI10.1214/18-AOS1717MaRDI QIDQ2414090zbMATH OpenWikidataFDO

Authors Niklas Willrich, V. G. Spokoiny

Publication date 10 May 2019

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1507.05034, https://projecteuclid.org/euclid.aos/1550026841

zbMATH Keywords

heteroscedasticitylinear regressionmodel selectionmultiple pairwise testsmultiplier bootstrapnonparametric regressionvariance co-variance structure

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Nonparametric regression and quantile regression (62G08) Nonparametric statistical resampling methods (62G09) Linear regression; mixed models (62J05) Analysis of variance and covariance (ANOVA) (62J10) Paired and multiple comparisons; multiple testing (62J15)

Abstract: In the problem of model selection for a given family of linear estimators, ordered by their variance, we offer a new "smallest accepted" approach motivated by Lepski's method and multiple testing theory. The procedure selects the smallest model which satisfies an acceptance rule based on comparison with all larger models. The method is completely data-driven and does not use any prior information about the variance structure of the noise: its parameters are adjusted to the underlying possibly heterogeneous noise by the so-called "propagation condition" using a wild bootstrap method. The validity of the bootstrap calibration is proved for finite samples with an explicit error bound. We provide a comprehensive theoretical study of the method and describe in detail the set of possible values of the selector ( hat{m} ). We also establish some precise oracle error bounds for the corresponding estimator ( hat{ heta} = ilde{ heta}_{hat{m}} ) which equally applies to estimation of the whole parameter vectors, some subvector or linear mapping, as well as the estimation of a linear functional.

Review text

To avoid overfitting or underfitting, algorithms for model selection are very important. The authors propose a procedure for linear regression with heteroscedastic Gaussian noise (that is assumed to be independent). The procedure is based on multiple pairwise tests and it selects the smallest model which is accepted when compared to all larger models. The authors prove an exponential bound for the difference of their procedure and the oracle estimator. As the variance covariance structure of the noise is unknown in practice, the authors suggest using the wild bootstrap with Gaussian multipliers. They show that this resampling method is consistent. The procedure is also evaluated in a simulations study about a nonparametric regression model. It shows that the bootstrap method has a similar performance as the procedure based on the knowledge of the variance structure.

Recommendations

• Bootstrap order selection for SETAR models
• Resampling-based tuning of ordered model selection
• Bootstrap Model Selection
• Bootstrap order selection for autoregressive models
• Bootstrap model selection for possibly dependent and heterogeneous data
• Bootstrap autoregressive order selection

Cited in

(4)

• Resampling-based tuning of ordered model selection
• Adaptive minimax optimality in statistical inverse problems via SOLIT—Sharp Optimal Lepskiĭ-Inspired Tuning
• Simple adaptive estimation of quadratic functionals in nonparametric IV models
• Adaptive estimation and uniform confidence bands for nonparametric structural functions and elasticities

This page was built for publication: Bootstrap tuning in Gaussian ordered model selection

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2414090)