Global and local two-sample tests via regression

DOI10.1214/19-EJS1648zbMATH Open1435.62199arXiv1812.08927MaRDI QIDQ2283577FDOQ2283577

Authors: Ilmun Kim, Ann B. Lee, Jing Lei 

Publication date: 3 January 2020

Published in: Electronic Journal of Statistics (Search for Journal in Brave)

Abstract: Two-sample testing is a fundamental problem in statistics. Despite its long history, there has been renewed interest in this problem with the advent of high-dimensional and complex data. Specifically, in the machine learning literature, there have been recent methodological developments such as classification accuracy tests. The goal of this work is to present a regression approach to comparing multivariate distributions of complex data. Depending on the chosen regression model, our framework can efficiently handle different types of variables and various structures in the data, with competitive power under many practical scenarios. Whereas previous work has been largely limited to global tests which conceal much of the local information, our approach naturally leads to a local two-sample testing framework in which we identify local differences between multivariate distributions with statistical confidence. We demonstrate the efficacy of our approach both theoretically and empirically, under some well-known parametric and nonparametric regression methods. Our proposed methods are applied to simulated data as well as a challenging astronomy data set to assess their practical usefulness.

Full work available at URL: https://arxiv.org/abs/1812.08927

Recommendations

• Two-sample Testing in High Dimensions
• Two-sample smooth tests for the equality of distributions
• Local significant differences from nonparametric two-sample tests
• A nonparametric two-sample test applicable to high dimensional data
• Optimal tests for the general two-sample problem

zbMATH Keywords

random forestskernel regressionpermutation testintrinsic dimensionnearest neighbor regressiongalaxy morphology

Mathematics Subject Classification ID

Nonparametric hypothesis testing (62G10) Asymptotic properties of nonparametric inference (62G20) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Linear regression; mixed models (62J05) Hypothesis testing in multivariate analysis (62H15) Image analysis in multivariate analysis (62H35) Applications of statistics to physics (62P35) Galactic and stellar structure (85A15)

Cites Work

• On the use of random forest for two-sample testing
• Statistics for high-dimensional data. Methods, theory and applications.
• Random forests
• High-dimensional generalized linear models and the lasso
• Diffusion maps
• A goodness-of-fit test for logistic regression models based on case-control data
• Title not available (Why is that?)
• Testing Statistical Hypotheses
• Comparing nonparametric versus parametric regression fits
• Two-sample test statistics for measuring discrepancies between two multivariate probability density functions using kernel-based density estimates
• Nonparametric smoothing and lack-of-fit tests
• On a new multivariate two-sample test.
• A sharper Bonferroni procedure for multiple tests of significance
• A kernel two-sample test
• All of Nonparametric Statistics
• A consistent test of functional form via nonparametric estimation techniques
• Comparing distributions
• Logistic disease incidence models and case-control studies
• Title not available (Why is that?)
• A distribution-free theory of nonparametric regression
• Non-asymptotic minimax rates of testing in signal detection
• An updated review of goodness-of-fit tests for regression models
• Separate sample logistic discrimination
• Geometric diffusions as a tool for harmonic analysis and structure definition of data: diffusion maps
• Analysis of a random forests model
• An empirical comparison of ensemble methods based on classification trees
• A power comparison between nonparametric regression tests.
• Testing the hypothesis of a general linear model using nonparametric regression estimation
• Information-theoretic determination of minimax rates of convergence
• Maximum likelihood for generalized case-control studies
• Testing a linear regression model against nonparametric alternatives
• Minimax Testing of Nonparametric Hypotheses on a Distribution Density in the $L_p$ Metrics
• Test of homogeneity in semiparametric two-sample density ratio models
• An estimate of the remainder in a combinatorial central limit theorem
• Rates of convergence for the \(k\)-nearest neighbor estimators with smoother regression functions
• On high dimensional two-sample tests based on nearest neighbors
• Lectures on the nearest neighbor method
• A review of 20 years of naive tests of significance for high-dimensional mean vectors and covariance matrices
• On robust estimation in logistic case-control studies
• Dimension reduction and variable selection in case control studies via regularized likelihood optimization
• Least-squares two-sample test
• Permutation tests for studying classifier performance
• Comparing two samples by penalized logistic regression
• High-order accurate methods for retrospective sampling problems
• Local significant differences from nonparametric two-sample tests
• Classification accuracy as a proxy for two-sample testing
• The classification permutation test: a flexible approach to testing for covariate imbalance in observational studies

Cited In (12)

• On the use of random forest for two-sample testing
• Model-independent detection of new physics signals using interpretable semisupervised classifier tests
• Minimax optimality of permutation tests
• A new set of tools for goodness-of-fit validation
• Local significant differences from nonparametric two-sample tests
• Globaltest confidence regions and their application to ridge regression
• Probabilistic multi-resolution scanning for two-sample differences
• Asymptotic Distribution-Free Independence Test for High-Dimension Data
• Testing under local misspecification and artificial regressions
• Detecting distributional differences in labeled sequence data with application to tropical cyclone satellite imagery
• ODC and ROC curves, comparison curves and stochastic dominance
• Exhaustive Goodness of Fit Via Smoothed Inference and Graphics

Uses Software

• GeneSrF
• hypoRF

This page was built for publication: Global and local two-sample tests via regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2283577)