Jackknife, bootstrap and other resampling methods in regression analysis (Q1822432): Difference between revisions

Statistical inference based on data resampling has been mostly based on the assumption of independence and identical distribution, the i.i.d. case. The author maintains that resampling methods justifiable in the i.i.d. case may not work in more complex situations. These methods are studied in the context of regression models. New methods are proposed that take into account special features of regression data. A class of weighted jackknife variance estimators for the least squares estimator, which deletes a fixed number of observations at a time, is proposed. The proposed estimator is unbiased for homoscedastic errors. The special (delete-one) jackknife is almost unbiased for heteroscedastic errors. The method is extended to cover nonlinear parameters, regression M-estimators, nonlinear regression and generalized linear models. Three bootstrap methods are considered. Two of them are biased variance estimators. A general method for resampling residuals is proposed. It gives variance estimators that are bias-robust.