p-values for classification

General considerations in statistical decision theory (62C05) Parametric tolerance and confidence regions (62F25) Nonparametric statistical resampling methods (62G09) Nonparametric tolerance and confidence regions (62G15) Classification and discrimination; cluster analysis (statistical aspects) (62H30)

Abstract: Let

(X, Y)

be a random variable consisting of an observed feature vector

X i n m a t h c a l X

and an unobserved class label

Y i n 1, 2, ..., L

with unknown joint distribution. In addition, let

m a t h c a l D

be a training data set consisting of

n

completely observed independent copies of

(X, Y)

. Usual classification procedures provide point predictors (classifiers)

w i d e h a t Y (X, m a t h c a l D)

of

Y

or estimate the conditional distribution of

Y

given

X

. In order to quantify the certainty of classifying

X

we propose to construct for each

h e t a = 1, 2, ..., L

a p-value

p i_{h e t a} (X, m a t h c a l D)

for the null hypothesis that

Y = h e t a

, treating

Y

temporarily as a fixed parameter. In other words, the point predictor

w i d e h a t Y (X, m a t h c a l D)

is replaced with a prediction region for

Y

with a certain confidence. We argue that (i) this approach is advantageous over traditional approaches and (ii) any reasonable classifier can be modified to yield nonparametric p-values. We discuss issues such as optimality, single use and multiple use validity, as well as computational and graphical aspects.

Recommendations

Cites work

Cited in

(3)

This page was built for publication: \(p\)-values for classification

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1951759)