Local nearest neighbour classification with applications to semi-supervised learning
From MaRDI portal
(Redirected from Publication:2196247)
Abstract: We derive a new asymptotic expansion for the global excess risk of a local--nearest neighbour classifier, where the choice of may depend upon the test point. This expansion elucidates conditions under which the dominant contribution to the excess risk comes from the decision boundary of the optimal Bayes classifier, but we also show that if these conditions are not satisfied, then the dominant contribution may arise from the tails of the marginal distribution of the features. Moreover, we prove that, provided the -dimensional marginal distribution of the features has a finite th moment for some (as well as other regularity conditions), a local choice of can yield a rate of convergence of the excess risk of , where is the sample size, whereas for the standard -nearest neighbour classifier, our theory would require and finite moments to achieve this rate. These results motivate a new -nearest neighbour classifier for semi-supervised learning problems, where the unlabelled data are used to obtain an estimate of the marginal feature density, and fewer neighbours are used for classification when this density estimate is small. Our worst-case rates are complemented by a minimax lower bound, which reveals that the local, semi-supervised -nearest neighbour classifier attains the minimax optimal rate over our classes for the excess risk, up to a subpolynomial factor in . These theoretical improvements over the standard -nearest neighbour classifier are also illustrated through a simulation study.
Recommendations
- Choice of neighbor order in nearest-neighbor classification
- An Optimal Global Nearest Neighbor Metric
- scientific article; zbMATH DE number 3879932
- Classification in general finite dimensional spaces with the \(k\)-nearest neighbor rule
- Asymptotic expansions of the \(k\) nearest neighbor risk
- Consistency of the \(k\)-nearest neighbors rule for functional data
- Nearest neighbor classification with dependent training sequences.
- Choice of the smoothing parameter and efficiency of k-nearest neighbor classification
- Lectures on the nearest neighbor method
Cites work
- scientific article; zbMATH DE number 5604036 (Why is no real title available?)
- scientific article; zbMATH DE number 893887 (Why is no real title available?)
- A Nonparametric Estimate of a Multivariate Density Function
- Bandwidth choice for nonparametric classification
- Choice of neighbor order in nearest-neighbor classification
- Classification in general finite dimensional spaces with the \(k\)-nearest neighbor rule
- Concentration inequalities. A nonasymptotic theory of independence
- Consistent nonparametric regression. Discussion
- Discriminatory Analysis. Nonparametric Discrimination: Consistency Properties
- Efficient multivariate entropy estimation via \(k\)-nearest neighbour distances
- Fast learning rates for plug-in classifiers
- Lectures on the nearest neighbor method
- Multivariate Two-Sample Tests Based on Nearest Neighbors
- Multivariate k-nearest neighbor density estimates
- Nearest neighbor pattern classification
- Nonparametric independence testing via mutual information
- On bandwidth variation in kernel estimates. A square root law
- On the rate of convergence of the bagged nearest neighbor estimate
- Optimal weighted nearest neighbour classifiers
- Rate of strong uniform convergence of k-NN density estimates
- Rates of convergence of nearest neighbor estimation under arbitrary sampling
- Rates of strong uniform consistency for multivariate kernel density estimators. (Vitesse de convergence uniforme presque sûre pour des estimateurs à noyaux de densités multivariées)
- Robust Subspace Clustering via Thresholding
- Sample estimate of the entropy of a random vector
- Smooth discrimination analysis
- Theoretical analysis of cross-validation for estimating the risk of the \(k\)-nearest neighbor classifier
- Theory of Classification: a Survey of Some Recent Advances
- Uniform asymptotics for kernel density estimators with variable bandwidths
- Variable Kernel Estimates of Multivariate Densities
Cited in
(17)- Nearest neighbor sampling for covariate shift adaptation
- Adaptive transfer learning
- Sharp-SSL: Selective High-Dimensional Axis-Aligned Random Projections for Semi-Supervised Learning
- Nonparametric estimation of surface integrals on level sets
- E. Fix and J.L. Hodges (1951): An Important Contribution to Nonparametric Discriminant Analysis and Density Estimation: Commentary on Fix and Hodges (1951)
- Local learning integrating global structure for large scale semi-supervised classification
- Locally nearest neighbor classifiers for pattern classification
- Asymptotics and optimal bandwidth for nonparametric estimation of density level sets
- Strongly universally consistent nonparametric regression and classification with privatised data
- Nearest neighbor empirical processes
- Benefit of Interpolation in Nearest Neighbor Algorithms
- A nearest-neighbor based nonparametric test for viral remodeling in heterogeneous single-cell proteomic data
- Marginal singularity and the benefits of labels in covariate-shift
- Theoretical analysis of cross-validation for estimating the risk of the \(k\)-nearest neighbor classifier
- An adaptive multiclass nearest neighbor classifier
- Robust transfer learning with unreliable source data
- Hybrid learning based on Fisher linear discriminant
This page was built for publication: Local nearest neighbour classification with applications to semi-supervised learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2196247)
