Optimal Nonparametric Inference with Two-Scale Distributional Nearest Neighbors
From MaRDI portal
Publication:6153993
Abstract: The weighted nearest neighbors (WNN) estimator has been popularly used as a flexible and easy-to-implement nonparametric tool for mean regression estimation. The bagging technique is an elegant way to form WNN estimators with weights automatically generated to the nearest neighbors; we name the resulting estimator as the distributional nearest neighbors (DNN) for easy reference. Yet, there is a lack of distributional results for such estimator, limiting its application to statistical inference. Moreover, when the mean regression function has higher-order smoothness, DNN does not achieve the optimal nonparametric convergence rate, mainly because of the bias issue. In this work, we provide an in-depth technical analysis of the DNN, based on which we suggest a bias reduction approach for the DNN estimator by linearly combining two DNN estimators with different subsampling scales, resulting in the novel two-scale DNN (TDNN) estimator. The two-scale DNN estimator has an equivalent representation of WNN with weights admitting explicit forms and some being negative. We prove that, thanks to the use of negative weights, the two-scale DNN estimator enjoys the optimal nonparametric rate of convergence in estimating the regression function under the fourth-order smoothness condition. We further go beyond estimation and establish that the DNN and two-scale DNN are both asymptotically normal as the subsampling scales and sample size diverge to infinity. For the practical implementation, we also provide variance estimators and a distribution estimator using the jackknife and bootstrap techniques for the two-scale DNN. These estimators can be exploited for constructing valid confidence intervals for nonparametric inference of the regression function. The theoretical results and appealing finite-sample performance of the suggested two-scale DNN method are illustrated with several numerical examples.
Cites work
- scientific article; zbMATH DE number 4102297 (Why is no real title available?)
- scientific article; zbMATH DE number 491591 (Why is no real title available?)
- scientific article; zbMATH DE number 952381 (Why is no real title available?)
- scientific article; zbMATH DE number 3053501 (Why is no real title available?)
- A Class of Statistics with Asymptotically Normal Distribution
- A bias-reduced approach to density estimation using Bernstein polynomials
- A distribution-free theory of nonparametric regression
- Approximating high-dimensional infinite-order \(U\)-statistics: statistical and computational guarantees
- Approximation Theorems of Mathematical Statistics
- Asymptotic Normality of Simple Linear Rank Statistics Under Alternatives
- Asymptotic distributions of high-dimensional distance correlation inference
- Bagging predictors
- Bias Reduction of Autoregressive Estimates in Time Series Regression Model through Restricted Maximum Likelihood
- Bootstrap methods: another look at the jackknife
- Effect of bias estimation on coverage accuracy of bootstrap confidence intervals for a probability density
- Efficient multivariate entropy estimation via \(k\)-nearest neighbour distances
- Estimation and Inference of Heterogeneous Treatment Effects using Random Forests
- Exact bootstrap \(k\)-nearest neighbor learners
- Generalized random forests
- High-dimensional classification using features annealed independence rules
- Improvement of Kernel Type Density Estimators
- Jackknifing $U$-Statistics
- Lectures on the nearest neighbor method
- Local Properties of k-NN Regression Estimates
- Measuring and testing dependence by correlation of distances
- NOTES ON BIAS IN ESTIMATION
- On Bias Reduction in Estimation
- On the Effect of Bias Estimation on Coverage Accuracy in Nonparametric Inference
- On the rate of convergence of the bagged nearest neighbor estimate
- Optimal global rates of convergence for nonparametric regression
- Optimal weighted nearest neighbour classifiers
- Properties of Bagged Nearest Neighbour Classifiers
- Random forests
- Sure independence screening for ultrahigh dimensional feature space. With discussion and authors' reply
- Twicing Kernels and a Small Bias Property of Semiparametric Estimators
This page was built for publication: Optimal Nonparametric Inference with Two-Scale Distributional Nearest Neighbors
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6153993)
