Optimal Nonparametric Inference with Two-Scale Distributional Nearest Neighbors

DOI10.1080/01621459.2022.2115375arXiv1808.08469OpenAlexW4292658306MaRDI QIDQ6153993FDOQ6153993

Authors: Emre Demirkaya, Yingying Fan, Lan Gao, Jinchi Lv, Patrick Vossler, Jingbo Wang 

Publication date: 19 March 2024

Published in: Journal of the American Statistical Association (Search for Journal in Brave)

Abstract: The weighted nearest neighbors (WNN) estimator has been popularly used as a flexible and easy-to-implement nonparametric tool for mean regression estimation. The bagging technique is an elegant way to form WNN estimators with weights automatically generated to the nearest neighbors; we name the resulting estimator as the distributional nearest neighbors (DNN) for easy reference. Yet, there is a lack of distributional results for such estimator, limiting its application to statistical inference. Moreover, when the mean regression function has higher-order smoothness, DNN does not achieve the optimal nonparametric convergence rate, mainly because of the bias issue. In this work, we provide an in-depth technical analysis of the DNN, based on which we suggest a bias reduction approach for the DNN estimator by linearly combining two DNN estimators with different subsampling scales, resulting in the novel two-scale DNN (TDNN) estimator. The two-scale DNN estimator has an equivalent representation of WNN with weights admitting explicit forms and some being negative. We prove that, thanks to the use of negative weights, the two-scale DNN estimator enjoys the optimal nonparametric rate of convergence in estimating the regression function under the fourth-order smoothness condition. We further go beyond estimation and establish that the DNN and two-scale DNN are both asymptotically normal as the subsampling scales and sample size diverge to infinity. For the practical implementation, we also provide variance estimators and a distribution estimator using the jackknife and bootstrap techniques for the two-scale DNN. These estimators can be exploited for constructing valid confidence intervals for nonparametric inference of the regression function. The theoretical results and appealing finite-sample performance of the suggested two-scale DNN method are illustrated with several numerical examples.

Full work available at URL: https://arxiv.org/abs/1808.08469

zbMATH Keywords

weighted nearest neighborsbagging\(k\)-nearest neighborsbootstrap and jackknifenonparametric estimation and inferencetwo-scale distributional nearest neighbors

Mathematics Subject Classification ID

Statistics (62-XX)

Cites Work

• Generalized random forests
• Measuring and testing dependence by correlation of distances
• Approximation Theorems of Mathematical Statistics
• Random forests
• Bagging predictors
• Bootstrap methods: another look at the jackknife
• Estimation and Inference of Heterogeneous Treatment Effects using Random Forests
• Sure Independence Screening for Ultrahigh Dimensional Feature Space
• High-dimensional classification using features annealed independence rules
• Jackknifing $U$-Statistics
• A Class of Statistics with Asymptotically Normal Distribution
• NOTES ON BIAS IN ESTIMATION
• Title not available (Why is that?)
• Optimal global rates of convergence for nonparametric regression
• Bias Reduction of Autoregressive Estimates in Time Series Regression Model through Restricted Maximum Likelihood
• Efficient multivariate entropy estimation via \(k\)-nearest neighbour distances
• Asymptotic Normality of Simple Linear Rank Statistics Under Alternatives
• Optimal weighted nearest neighbour classifiers
• A distribution-free theory of nonparametric regression
• On the Effect of Bias Estimation on Coverage Accuracy in Nonparametric Inference
• A bias-reduced approach to density estimation using Bernstein polynomials
• Effect of bias estimation on coverage accuracy of bootstrap confidence intervals for a probability density
• Title not available (Why is that?)
• Twicing Kernels and a Small Bias Property of Semiparametric Estimators
• Title not available (Why is that?)
• Improvement of Kernel Type Density Estimators
• On Bias Reduction in Estimation
• Local Properties of k-NN Regression Estimates
• Lectures on the nearest neighbor method
• Exact bootstrap \(k\)-nearest neighbor learners
• Properties of Bagged Nearest Neighbour Classifiers
• On the rate of convergence of the bagged nearest neighbor estimate
• Title not available (Why is that?)
• Asymptotic distributions of high-dimensional distance correlation inference
• Approximating high-dimensional infinite-order \(U\)-statistics: statistical and computational guarantees

This page was built for publication: Optimal Nonparametric Inference with Two-Scale Distributional Nearest Neighbors

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6153993)