Imputation Scores
From MaRDI portal
Abstract: Given the prevalence of missing data in modern statistical research, a broad range of methods is available for any given imputation task. How does one choose the `best' imputation method in a given application? The standard approach is to select some observations, set their status to missing, and compare prediction accuracy of the methods under consideration of these observations. Besides having to somewhat artificially mask observations, a shortcoming of this approach is that imputations based on the conditional mean will rank highest if predictive accuracy is measured with quadratic loss. In contrast, we want to rank highest an imputation that can sample from the true conditional distributions. In this paper, we develop a framework called "Imputation Scores" (I-Scores) for assessing missing value imputations. We provide a specific I-Score based on density ratios and projections, that is applicable to discrete and continuous data. It does not require to mask additional observations for evaluations and is also applicable if there are no complete observations. The population version is shown to be proper in the sense that the highest rank is assigned to an imputation method that samples from the correct conditional distribution. The propriety is shown under the missing completely at random (MCAR) assumption but is also shown to be valid under missing at random (MAR) with slightly more restrictive assumptions. We show empirically on a range of data sets and imputation methods that our score consistently ranks true data high(est) and is able to avoid pitfalls usually associated with performance measures such as RMSE. Finally, we provide the R-package Iscores available on CRAN with an implementation of our method.
Cites work
- scientific article; zbMATH DE number 3860199 (Why is no real title available?)
- scientific article; zbMATH DE number 4088699 (Why is no real title available?)
- scientific article; zbMATH DE number 893887 (Why is no real title available?)
- scientific article; zbMATH DE number 3252891 (Why is no real title available?)
- A comparison of imputation methods for handling missing scores in biometric fusion
- A general theory for jackknife variance estimation
- A nonparametric test of missing completely at random for incomplete multivariate data
- Analyzing bagging
- Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds
- BEST: a decision tree algorithm that handles missing values
- Flexible imputation of missing data
- Handling missing values when applying classification models
- Making and evaluating point forecasts
- Multiple imputation for continuous variables using a Bayesian principal component analysis
- Multiple imputation in principal component analysis
- Node harvest
- On the empirical estimation of integral probability metrics
- Random forests
- Recursive partitioning for missing data imputation in the presence of interaction effects
- Strictly Proper Scoring Rules, Prediction, and Estimation
- Two‐sample test based on classification probability
- Using Proper Divergence Functions to Evaluate Climate Models
- What is meant by ``missing at random?
This page was built for publication: Imputation Scores
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q116910)
