Distribution-free tests for sparse heterogeneous mixtures

DOI10.1007/S11749-016-0499-XzbMATH Open1422.62159arXiv1308.0346OpenAlexW1781020620MaRDI QIDQ2398079FDOQ2398079

Authors: Ery Arias-Castro, Meng Wang 

Publication date: 15 August 2017

Published in: Test (Search for Journal in Brave)

Abstract: We consider the problem of detecting sparse heterogeneous mixtures from a nonparametric perspective, and develop distribution-free tests when all effects have the same sign. Specifically, we assume that the null distribution is symmetric about zero, while the true effects have positive median. We evaluate the precise performance of classical tests for the median (t-test, sign test) and classical tests for symmetry (signed-rank, Smirnov, total number of runs, longest run tests) showing that none of them is asymptotically optimal for the normal mixture model in all sparsity regimes. We then suggest two new tests. The main one is a form of Higher Criticism, or Anderson-Darling, test for symmetry. It is shown to be asymptotically optimal for the normal mixture model, and other generalized Gaussian mixture models, in all sparsity regimes. Our numerical experiments confirm our theoretical findings.

Full work available at URL: https://arxiv.org/abs/1308.0346

Recommendations

• Higher criticism for detecting sparse heterogeneous mixtures.
• Optimal detection of heterogeneous and heteroscedastic mixtures
• Detection of sparse mixtures: higher criticism and scan statistic
• The sparse variance contamination model
• Non-asymptotic detection of two-component mixtures with unknown means

zbMATH Keywords

distribution-free testshigher criticismAnderson-Darling testmixture detectionSmirnov test for symmetry

Mathematics Subject Classification ID

Nonparametric hypothesis testing (62G10) Asymptotic properties of nonparametric inference (62G20) Statistics of extreme values; tail inference (62G32)

Cites Work

• Detection boundary in sparse regression
• Testing Statistical Hypotheses
• Global testing under sparse alternatives: ANOVA, multiple comparisons and the higher criticism
• Some problems of hypothesis testing leading to infinitely divisible distributions
• Higher criticism for detecting sparse heterogeneous mixtures.
• Goodness-of-fit tests via phi-divergences
• Properties of higher criticism under strong dependence
• Asymptotic Theory of Certain "Goodness of Fit" Criteria Based on Stochastic Processes
• A limit theorem for the maximum of normalized sums of independent random variables
• Innovated higher criticism for detecting sparse signals in correlated noise
• Multiple testing procedures with applications to genomics.
• Optimal detection of heterogeneous and heteroscedastic mixtures
• Title not available (Why is that?)
• Robustness and Accuracy of Methods for High Dimensional Data Analysis Based on Student’s t-Statistic
• Distribution-free tests for sparse heterogeneous mixtures
• Optimal Detection of Sparse Mixtures Against a Given Null Distribution
• Adaptive detection of a signal of growing dimension. II
• Adaptive detection of a signal of growing dimension. I
• TESTING SYMMETRY USING A TRIMMED LONGEST RUN STATISTIC

Cited In (24)

• Testing independent sparse heterogeneous mixtures
• Beyond HC: more sensitive tests for rare/weak alternatives
• Higher criticism for detecting sparse heterogeneous mixtures.
• Supplementary score test for sparse signals in large-scale truncated one-sided sequential tests
• Detection and feature selection in sparse mixture models
• Anomaly Detection for a Large Number of Streams: A Permutation-Based Higher Criticism Approach
• Optimal detection of heterogeneous and heteroscedastic mixtures
• Detecting sparse heterogeneous mixtures in a two-sample problem
• AdaReg: data adaptive robust estimation in linear regression with application in GTEx gene expressions
• Optimal detection of weak positive latent dependence between two sequences of multiple tests
• Signal detection via Phi-divergences for general mixtures
• Signal-noise ratio of genetic associations and statistical power of SNP-set tests
• Distribution-free tests for sparse heterogeneous mixtures
• The impossibility region for detecting sparse mixtures using the higher criticism
• Detection boundary for a sparse gamma scale mixture model
• \(L_2\)-tests for sparse multinomials
• Optimal Detection of Sparse Mixtures Against a Given Null Distribution
• Higher criticism for large-scale inference, especially for rare and weak effects
• Detection of sparse positive dependence
• Optimal detection of multi-sample aligned sparse signals
• Consistency of the generalized likelihood ratio test for heteroscedastic normal mixtures
• Detectability of nonparametric signals: higher criticism versus likelihood ratio
• Detection of sparse mixtures: higher criticism and scan statistic
• The sparse Poisson means model

This page was built for publication: Distribution-free tests for sparse heterogeneous mixtures

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2398079)