Measuring reproducibility of high-throughput experiments

DOI10.1214/11-AOAS466zbMATH Open1231.62124arXiv1110.4705OpenAlexW3104066961WikidataQ112778340 ScholiaQ112778340MaRDI QIDQ652341FDOQ652341

Authors: Qunhua Li, James B. Brown, Haiyan Huang, P. J. Bickel 

Publication date: 14 December 2011

Published in: The Annals of Applied Statistics (Search for Journal in Brave)

Abstract: Reproducibility is essential to reliable scientific discovery in high-throughput experiments. In this work we propose a unified approach to measure the reproducibility of findings identified from replicate experiments and identify putative discoveries using reproducibility. Unlike the usual scalar measures of reproducibility, our approach creates a curve, which quantitatively assesses when the findings are no longer consistent across replicates. Our curve is fitted by a copula mixture model, from which we derive a quantitative reproducibility score, which we call the "irreproducible discovery rate" (IDR) analogous to the FDR. This score can be computed at each set of paired replicate ranks and permits the principled setting of thresholds both for assessing reproducibility and combining replicates. Since our approach permits an arbitrary scale for each replicate, it provides useful descriptive measures in a wide variety of situations to be explored. We study the performance of the algorithm using simulations and give a heuristic analysis of its theoretical properties. We demonstrate the effectiveness of our method in a ChIP-seq experiment.

Full work available at URL: https://arxiv.org/abs/1110.4705

Recommendations

• Maximum rank reproducibility: a nonparametric approach to assessing reproducibility in replicate experiments
• A regression framework for assessing covariate effects on the reproducibility of high-throughput experiments
• Posterior probability of discovery and expected rate of discovery for multiple hypothesis testing and high throughput assays
• Segmented Correspondence Curve Regression for Quantifying Covariate Effects on the Reproducibility of High-Throughput Experiments
• Improving reproducibility probability estimation and preserving \textit{RP}-testing

zbMATH Keywords

copulaiterative algorithmassociationmixture modelgenomicsirreproducible discovery rate

Mathematics Subject Classification ID

Multivariate analysis (62H99) Applications of statistics to biology and medical sciences; meta analysis (62P10) Graphical methods in statistics (62A09)

Cites Work

• Measuring reproducibility of high-throughput experiments
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Multivariate survival distributions
• Chi-plots for assessing dependence
• Title not available (Why is that?)
• On blest's measure of rank correlation
• A Direct Approach to False Discovery Rates
• A semiparametric estimation procedure of dependence parameters in multivariate families of distributions
• Detecting dependence with Kendall plots
• Theory & Methods: Rank Correlation — an Alternative Measure
• Oracle and Adaptive Compound Decision Rules for False Discovery Rate Control
• Title not available (Why is that?)
• The positive false discovery rate: A Bayesian interpretation and the \(q\)-value
• Data-Driven Rank Tests for Independence
• Operating Characteristics and Extensions of the False Discovery Rate Procedure
• A WEIGHTED RANK MEASURE OF CORRELATION

Cited In (16)

• Deriving chemosensitivity from cell lines: forensic bioinformatics and reproducible research in high-throughput biology
• Reproducibility of biomarker identifications from mass spectrometry proteomic data in cancer studies
• Maximum rank reproducibility: a nonparametric approach to assessing reproducibility in replicate experiments
• Discovering findings that replicate from a primary study of high dimension to a follow-up study
• A Survey of Reporting Practices of Computer Simulation Studies in Statistical Research
• A regression framework for assessing covariate effects on the reproducibility of high-throughput experiments
• Assessing replicability of findings across two studies of multiple features
• Assessing reproducibility of high-throughput experiments in the case of missing data
• Challenges for machine learning in RNA-protein interaction prediction
• idr
• Segmented Correspondence Curve Regression for Quantifying Covariate Effects on the Reproducibility of High-Throughput Experiments
• Overall assessment for selected markers from high-throughput data
• Measuring reproducibility of high-throughput experiments
• On the exact region determined by Spearman's \(\rho\) and Blest's measure of rank correlation \(\nu\) for bivariate extreme-value copulas
• Detection of sparse positive dependence
• Reproducible Research: A Bioinformatics Case Study

Uses Software

• F-Seq

This page was built for publication: Measuring reproducibility of high-throughput experiments

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q652341)