Measuring reproducibility of high-throughput experiments
From MaRDI portal
Publication:652341
Abstract: Reproducibility is essential to reliable scientific discovery in high-throughput experiments. In this work we propose a unified approach to measure the reproducibility of findings identified from replicate experiments and identify putative discoveries using reproducibility. Unlike the usual scalar measures of reproducibility, our approach creates a curve, which quantitatively assesses when the findings are no longer consistent across replicates. Our curve is fitted by a copula mixture model, from which we derive a quantitative reproducibility score, which we call the "irreproducible discovery rate" (IDR) analogous to the FDR. This score can be computed at each set of paired replicate ranks and permits the principled setting of thresholds both for assessing reproducibility and combining replicates. Since our approach permits an arbitrary scale for each replicate, it provides useful descriptive measures in a wide variety of situations to be explored. We study the performance of the algorithm using simulations and give a heuristic analysis of its theoretical properties. We demonstrate the effectiveness of our method in a ChIP-seq experiment.
Recommendations
- Maximum rank reproducibility: a nonparametric approach to assessing reproducibility in replicate experiments
- A regression framework for assessing covariate effects on the reproducibility of high-throughput experiments
- Posterior probability of discovery and expected rate of discovery for multiple hypothesis testing and high throughput assays
- Segmented Correspondence Curve Regression for Quantifying Covariate Effects on the Reproducibility of High-Throughput Experiments
- Improving reproducibility probability estimation and preserving \textit{RP}-testing
Cites work
- scientific article; zbMATH DE number 3163305 (Why is no real title available?)
- scientific article; zbMATH DE number 3656971 (Why is no real title available?)
- scientific article; zbMATH DE number 3567782 (Why is no real title available?)
- scientific article; zbMATH DE number 720689 (Why is no real title available?)
- scientific article; zbMATH DE number 1134711 (Why is no real title available?)
- scientific article; zbMATH DE number 5058453 (Why is no real title available?)
- A Direct Approach to False Discovery Rates
- A WEIGHTED RANK MEASURE OF CORRELATION
- A semiparametric estimation procedure of dependence parameters in multivariate families of distributions
- Chi-plots for assessing dependence
- Data-Driven Rank Tests for Independence
- Detecting dependence with Kendall plots
- Measuring reproducibility of high-throughput experiments
- Multivariate survival distributions
- On blest's measure of rank correlation
- Operating Characteristics and Extensions of the False Discovery Rate Procedure
- Oracle and Adaptive Compound Decision Rules for False Discovery Rate Control
- The positive false discovery rate: A Bayesian interpretation and the \(q\)-value
- Theory & Methods: Rank Correlation — an Alternative Measure
Cited in
(16)- A Survey of Reporting Practices of Computer Simulation Studies in Statistical Research
- Assessing reproducibility of high-throughput experiments in the case of missing data
- Challenges for machine learning in RNA-protein interaction prediction
- On the exact region determined by Spearman's \(\rho\) and Blest's measure of rank correlation \(\nu\) for bivariate extreme-value copulas
- Reproducibility of biomarker identifications from mass spectrometry proteomic data in cancer studies
- A regression framework for assessing covariate effects on the reproducibility of high-throughput experiments
- Assessing replicability of findings across two studies of multiple features
- Measuring reproducibility of high-throughput experiments
- idr
- Reproducible Research: A Bioinformatics Case Study
- Segmented Correspondence Curve Regression for Quantifying Covariate Effects on the Reproducibility of High-Throughput Experiments
- Deriving chemosensitivity from cell lines: forensic bioinformatics and reproducible research in high-throughput biology
- Maximum rank reproducibility: a nonparametric approach to assessing reproducibility in replicate experiments
- Overall assessment for selected markers from high-throughput data
- Discovering findings that replicate from a primary study of high dimension to a follow-up study
- Detection of sparse positive dependence
This page was built for publication: Measuring reproducibility of high-throughput experiments
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q652341)
