Optimal cross-validation in density estimation with the L^2-loss

DOI10.1214/14-AOS1240MaRDI QIDQ480972zbMATH OpenWikidataFDO

Authors Alain Celisse

Publication date 12 December 2014

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0811.0802

zbMATH Keywords

concentration inequalitiescross-validationdensity estimationleave-\(p\)-outmodel selectionoracle inequalityprojection estimatorsresamplingrisk estimation

Mathematics Subject Classification ID

Approximations to statistical distributions (nonasymptotic) (62E17) Density estimation (62G07) Nonparametric statistical resampling methods (62G09)

Abstract: We analyze the performance of cross-validation (CV) in the density estimation framework with two purposes: (i) risk estimation and (ii) model selection. The main focus is given to the so-called leave-$p$-out CV procedure (Lpo), where $p$ denotes the cardinality of the test set. Closed-form expressions are settled for the Lpo estimator of the risk of projection estimators. These expressions provide a great improvement upon $V$-fold cross-validation in terms of variability and computational complexity. From a theoretical point of view, closed-form expressions also enable to study the Lpo performance in terms of risk estimation. The optimality of leave-one-out (Loo), that is Lpo with $p=1$, is proved among CV procedures used for risk estimation. Two model selection frameworks are also considered: estimation, as opposed to identification. For estimation with finite sample size $n$, optimality is achieved for $p$ large enough [with $p/n=o(1)$] to balance the overfitting resulting from the structure of the model collection. For identification, model selection consistency is settled for Lpo as long as $p/n$ is conveniently related to the rate of convergence of the best estimator in the collection: (i) $p/no1$ as $no+infty$ with a parametric rate, and (ii) $p/n=o(1)$ with some nonparametric estimators. These theoretical results are validated by simulation experiments.

Recommendations

• Nonparametric density estimation by exact leave-\(p\)-out cross-validation
• Asymptotics of cross-validated risk estimation in estimator selection and performance assess\-ment
• A survey of cross-validation procedures for model selection
• scientific article; zbMATH DE number 2062404
• A local cross-validation algorithm

Cited in

(16)

• Nonparametric density estimation by exact leave-\(p\)-out cross-validation
• Extent to which least-squares cross-validation minimises integrated square error in nonparametric density estimation
• Targeted cross-validation
• Penalized unimodal spline density estimation with application to \(M\)-estimation
• The leave-worst-k-out criterion for cross validation
• Choice of V for V-Fold Cross-Validation in Least-Squares Density Estimation
• The cross-validated adaptive epsilon-net estimator
• scientific article; zbMATH DE number 2062404 (Why is no real title available?)
• Theoretical analysis of cross-validation for estimating the risk of the \(k\)-nearest neighbor classifier
• An efficient variance estimator for cross-validation under partition sampling
• Exact cross-validation for kNN: application to passive and active learning in classification
• Concentration inequalities of the cross-validation estimator for empirical risk minimizer
• Estimating the Kullback–Liebler risk based on multifold cross‐validation
• Fast and fully-automated histograms for large-scale data sets
• Weighted leave-one-out cross validation
• Deconvolution density estimation using penalized splines

This page was built for publication: Optimal cross-validation in density estimation with the \(L^{2}\)-loss

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q480972)