Properties of higher criticism under strong dependence
From MaRDI portal
Publication:2477065
Abstract: The problem of signal detection using sparse, faint information is closely related to a variety of contemporary statistical problems, including the control of false-discovery rate, and classification using very high-dimensional data. Each problem can be solved by conducting a large number of simultaneous hypothesis tests, the properties of which are readily accessed under the assumption of independence. In this paper we address the case of dependent data, in the context of higher criticism methods for signal detection. Short-range dependence has no first-order impact on performance, but the situation changes dramatically under strong dependence. There, although higher criticism can continue to perform well, it can be bettered using methods based on differences of signal values or on the maximum of the data. The relatively inferior performance of higher criticism in such cases can be explained in terms of the fact that, under strong dependence, the higher criticism statistic behaves as though the data were partitioned into very large blocks, with all but a single representative of each block being eliminated from the dataset.
Recommendations
- On strong invariance principles under dependence assumptions
- Strong independence and its spectrum
- On extremal dependence: some contributions
- On exact strong laws of large numbers under general dependence conditions
- On the properties of extended inclusion dependences
- Probability inequalities for certain dependence structures
- Strong invariance principles for dependent random variables
Cites work
- A stochastic process approach to false discovery control.
- Asymptotic minimaxity of false discovery rate thresholding for sparse exponential data
- Basic concepts of multiple tests -- a survey
- Cosmological non-Gaussian signature detection: comparing performance of different statistical tests
- Detecting a target in very noisy data from mutliple looks
- Empirical Bayes Analysis of a Microarray Experiment
- Estimating the Null and the Proportion of Nonnull Effects in Large-Scale Multiple Comparisons
- Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses
- Estimation and confidence sets for sparse normal mixtures
- Global and multiple test procedures using ordered \(p\)-values -- a review
- Goodness-of-fit tests via phi-divergences
- Higher criticism for detecting sparse heterogeneous mixtures.
- scientific article; zbMATH DE number 50804 (Why is no real title available?)
- scientific article; zbMATH DE number 720689 (Why is no real title available?)
- scientific article; zbMATH DE number 3303655 (Why is no real title available?)
- Minimax detection of a signal for \(l^ n\)-balls.
- Multiple hypothesis testing in microarray experiments.
- Some problems of hypothesis testing leading to infinitely divisible distributions
- Testing Statistical Hypotheses
- The limiting behavior of a modified maximal symmetric 2s-spacing with applications
- The optimal discovery procedure for large-scale significance testing, with applications to comparative microarray experiments
- Theoretical Measures of Relative Performance of Classifiers for High Dimensional Data with Small Sample Sizes
Cited in
(35)- Two sample tests for high-dimensional covariance matrices
- Sharp fixed \(n\) bounds and asymptotic expansions for the mean and the median of a Gaussian sample maximum, and applications to the Donoho-Jin model
- Detecting rare and faint signals via thresholding maximum likelihood estimators
- Global testing against sparse alternatives under Ising models
- A two-sample test for the equality of univariate marginal distributions for high-dimensional data
- A rate optimal procedure for recovering sparse differences between high-dimensional means under dependence
- Optimal inference with a multidimensional multiscale statistic
- Testing regression coefficients in high-dimensional and sparse settings
- Discovering focal regions of slightly-aggregated sparse signals
- Variable selection via adaptive false negative control in linear regression
- Sign-based test for mean vector in high-dimensional and sparse settings
- Distribution-free tests for sparse heterogeneous mixtures
- Optimal detection of weak positive latent dependence between two sequences of multiple tests
- Detection of an anomalous cluster in a network
- Effects of statistical dependence on multiple testing under a hidden Markov model
- Tests alternative to higher criticism for high-dimensional means under sparsity and column-wise dependence
- Higher criticism for large-scale inference, especially for rare and weak effects
- Goodness of fit tests in terms of local levels with special emphasis on higher criticism tests
- The intermediates take it all: asymptotics of higher criticism statistics and a powerful alternative based on equal local levels
- Asymptotics of goodness-of-fit tests based on minimum p-value statistics
- Optimal detection of heterogeneous and heteroscedastic mixtures
- Feature selection by higher criticism thresholding achieves the optimal phase diagram
- Detection boundary and higher criticism approach for rare and weak genetic effects
- Asymptotic optimality of the Westfall-Young permutation procedure for multiple testing under dependence
- Statistical proof? The problem of irreproducibility
- Diagonally Dominant Principal Component Analysis
- Detecting structured signals in Ising models
- Anomaly Detection for a Large Number of Streams: A Permutation-Based Higher Criticism Approach
- Global testing under sparse alternatives: ANOVA, multiple comparisons and the higher criticism
- Sharp multiple testing boundary for sparse sequences
- Controlling the number of significant effects in multiple testing
- Sparse anomaly detection across referentials: a rank-based higher criticism approach
- Robust detection of watermarks for large language models under human edits
- An adaptive decorrelation procedure for signal detection
- Innovated higher criticism for detecting sparse signals in correlated noise
This page was built for publication: Properties of higher criticism under strong dependence
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2477065)
