A Bayesian Reassessment of Nearest-Neighbor Classification

DOI10.1198/JASA.2009.0125MaRDI QIDQ5256123zbMATH OpenOpenAlexWikidataFDO

Authors Lionel Cucala, Jean-Michel Marin, Christian P. Robert Robert, D. M. Titterington

Publication date 22 June 2015

Published in Journal of the American Statistical Association (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0802.1357, https://hal.inria.fr/inria-00143783v4/file/RR-6173.pdf

zbMATH Keywords

normalizing constantpseudo-likelihoodpath samplingMarkov chain Monte Carlo algorithmperfect samplingBoltzmann modelcompatible conditionals

Mathematics Subject Classification ID

Bayesian inference (62F15) Classification and discrimination; cluster analysis (statistical aspects) (62H30)

Abstract: The k-nearest-neighbour procedure is a well-known deterministic method used in supervised classification. This paper proposes a reassessment of this approach as a statistical technique derived from a proper probabilistic model; in particular, we modify the assessment made in a previous analysis of this method undertaken by Holmes and Adams (2002,2003), and evaluated by Manocha and Girolami (2007), where the underlying probabilistic model is not completely well-defined. Once a clear probabilistic basis for the k-nearest-neighbour procedure is established, we derive computational tools for conducting Bayesian inference on the parameters of the corresponding model. In particular, we assess the difficulties inherent to pseudo-likelihood and to path sampling approximations of an intractable normalising constant, and propose a perfect sampling strategy to implement a correct MCMC sampler associated with our model. If perfect sampling is not available, we suggest using a Gibbs sampling approximation. Illustrations of the performance of the corresponding Bayesian classifier are provided for several benchmark datasets, demonstrating in particular the limitations of the pseudo-likelihood approximation in this set-up.

Recommendations

• The Nearest Neighbor and the Bayes Error Rates
• Bayesian adaptive nearest neighbor
• A Bayesian decision theory approach to classification problems
• A Bayesian Analysis of Recent Developments in Pattern Classification
• An efficient algorithm for Bayesian nearest neighbours
• scientific article; zbMATH DE number 2088082
• Likelihood inference in nearest-neighbour classification models

Cited in

(18)

• Scalable Bayesian inference for the inverse temperature of a hidden Potts model
• Estimating the Posterior Probabilities Using the K-Nearest Neighbor Rule
• Pseudo-likelihood inference underestimates model uncertainty: evidence from Bayesian nearest neighbours
• Bayesian computation for statistical models with intractable normalizing constants
• Spatial quantile clustering of climate data
• On the theoretical properties of the exchange algorithm
• Bayesian computation: a summary of the current state, and samples backwards and forwards
• A Bayesian Analysis of Recent Developments in Pattern Classification
• Likelihood inference in nearest-neighbour classification models
• scientific article; zbMATH DE number 4163939 (Why is no real title available?)
• The Nearest Neighbor and the Bayes Error Rates
• Boosting k-NN for categorization of natural scenes
• An efficient algorithm for Bayesian nearest neighbours
• Approximate Bayesian computational methods
• Properties of Bagged Nearest Neighbour Classifiers
• Pre-processing for approximate Bayesian computation in image analysis
• Efficient MCMC for Gibbs random fields using pre-computation
• A similarity-based Bayesian mixture-of-experts model

This page was built for publication: A Bayesian Reassessment of Nearest-Neighbor Classification

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5256123)