Testing for outliers with conformal p-values
From MaRDI portal
Abstract: This paper studies the construction of p-values for nonparametric outlier detection, taking a multiple-testing perspective. The goal is to test whether new independent samples belong to the same distribution as a reference data set or are outliers. We propose a solution based on conformal inference, a broadly applicable framework which yields p-values that are marginally valid but mutually dependent for different test points. We prove these p-values are positively dependent and enable exact false discovery rate control, although in a relatively weak marginal sense. We then introduce a new method to compute p-values that are both valid conditionally on the training data and independent of each other for different test points; this paves the way to stronger type-I error guarantees. Our results depart from classical conformal inference as we leverage concentration inequalities rather than combinatorial arguments to establish our finite-sample guarantees. Furthermore, our techniques also yield a uniform confidence bound for the false positive rate of any outlier detection algorithm, as a function of the threshold applied to its raw statistics. Finally, the relevance of our results is demonstrated by numerical experiments on real and simulated data.
Recommendations
Cites work
- 400: A Method for Combining Non-Independent, One-Sided Tests of Significance
- A conformal prediction approach to explore functional data
- A Direct Approach to False Discovery Rates
- Adaptive linear step-up procedures that control the false discovery rate
- An Extension of Wilks' Method for Setting Tolerance Limits
- An improved Bonferroni procedure for multiple tests of significance
- Asymptotic Minimax Character of the Sample Distribution Function and of the Classical Multinomial Estimator
- Classification accuracy as a proxy for two-sample testing
- Combining dependent P-values.
- Cross-conformal predictors
- Distribution-free, Risk-controlling Prediction Sets
- Finding an unknown number of multivariate outliers
- Generalized D^+_n Statistics
- Generalizing Simes' test and Hochberg's stepup procedure
- scientific article; zbMATH DE number 3684698 (Why is no real title available?)
- scientific article; zbMATH DE number 720689 (Why is no real title available?)
- scientific article; zbMATH DE number 2168212 (Why is no real title available?)
- scientific article; zbMATH DE number 1931847 (Why is no real title available?)
- scientific article; zbMATH DE number 3008134 (Why is no real title available?)
- scientific article; zbMATH DE number 7370598 (Why is no real title available?)
- scientific article; zbMATH DE number 3208674 (Why is no real title available?)
- Inductive conformal anomaly detection for sequential detection of anomalous sub-trajectories
- Multiple testing with the structure-adaptive Benjamini-Hochberg algorithm
- Multivariate outlier detection with high-breakdown estimators
- Non-Parametric Estimation II. Statistically Equivalent Blocks and Tolerance Regions--The Continuous Case
- Prediction and Outlier Detection in Classification Problems
- Predictive inference with the jackknife+
- Robustness of multiple testing procedures against dependence
- Scikit-learn: machine learning in Python
- Semi-supervised multiple testing
- Statistical Prediction with Special Reference to the Problem of Tolerance Limits
- Statistical Tolerance Regions: Theory, Applications, and Computation
- Strong Control, Conservative Point Estimation and Simultaneous Conservative Consistency of False Discovery Rates: A Unified Approach
- Testing randomness online
- The asymptotic distribution of the suprema of the standardized empirical processes
- The control of the false discovery rate in multiple testing under dependency.
- The harmonic mean p-value for combining dependent tests
- The limits of distribution-free conditional predictive inference
- The Simes Method for Multiple Hypothesis Testing With Positively Dependent Test Statistics
- The tight constant in the Dvoretzky-Kiefer-Wolfowitz inequality
Cited in
(26)- Testing normality in the presence of outliers
- A Scale-Free Approach for False Discovery Rate Control in Generalized Linear Models
- Conformal prediction beyond exchangeability
- Adaptive novelty detection with false discovery rate guarantee
- Testing with \(\mathrm{p}^*\)-values: between p-values, mid p-values, and e-values
- A confidence machine for sparse high-order interaction model
- Invariant correlation under marginal transforms
- A locally adaptive shrinkage approach to false selection rate control in high-dimensional classification
- Optimal Subsampling via Predictive Inference
- Fair conformal prediction for incomplete covariate data
- SymmPI: predictive inference for data with group symmetries
- Selecting informative conformal prediction sets with false coverage rate control
- Adaptive conformal classification with noisy labels
- An adaptive null proportion estimator for false discovery rate control
- Conformal inference for random objects
- Learn then test: calibrating predictive algorithms to achieve risk control
- False Discovery Rate Control For Structured Multiple Testing: Asymmetric Rules And Conformal Q -values
- Conformal link prediction for false discovery rate control
- Model-free selective inference under covariate shift via weighted conformal p-values
- When Tukey Meets Chauvenet: A New Boxplot Criterion for Outlier Detection
- Subuniformity of harmonic mean p-values
- Finding distributions that differ, with false discovery rate control
- Auxiliary data enhanced conformal prediction
- Conformal outlier detection for multivariate functional data
- A distribution-free EWMA control chart based on conformal p-values
- clis
This page was built for publication: Testing for outliers with conformal p-values
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6046306)
