A statistical approach to calibrating the scores of biased reviewers of scientific papers

DOI10.1007/S00184-015-0542-ZMaRDI QIDQ267651zbMATH OpenOpenAlexFDO

Authors Wiltrud Kuhlisch, Magnus Roos, Jörg Rothe, J. Rudolph, Björn Scheuermann, Dietrich Stoyan

Publication date 11 April 2016

Published in Metrika (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1007/s00184-015-0542-z

zbMATH Keywords

analysis of variancedesign matrixmodel with main effectspeer reviewingreview scores

Mathematics Subject Classification ID

Analysis of variance and covariance (ANOVA) (62J10) Applications of statistics (62P99) Statistical methods; economic indices and measures (91B82)

Recommendations

• Creating a consensus ranking of proposals from reviewers' partial ordinal rankings
• Statistical auditing of non-transparent expert assessments
• An alternative polynomial-sized formulation and an optimization based heuristic for the reviewer assignment problem
• Assigning papers to referees
• Design and analysis of the NIPS 2016 review process

Cites work

• A Richer Understanding of the Complexity of Election Systems
• A Unified Nonparametric Approach for Unbalanced Factorial Designs
• Bayesian analysis of order-statistics models for ranking data
• Computational Aspects of Approval Voting
• scientific article; zbMATH DE number 3883474 (Why is no real title available?)
• scientific article; zbMATH DE number 5080660 (Why is no real title available?)
• scientific article; zbMATH DE number 1315210 (Why is no real title available?)
• scientific article; zbMATH DE number 1156422 (Why is no real title available?)
• scientific article; zbMATH DE number 847276 (Why is no real title available?)
• Influence analysis of ranking data
• Probability models on rankings
• Statistical methods for ranking data

Cited in

(5)

• Quantitative measurements of the influence of participant roles during peer review meetings
• Review times in statistical journals: tilting at windmills?
• Design and analysis of the NIPS 2016 review process
• scientific article; zbMATH DE number 7415106 (Why is no real title available?)
• Refinement: Measuring informativeness of ratings in the absence of a gold standard

Describes a project that uses

Uses Software

• WWGbook
• BIOMstat

This page was built for publication: A statistical approach to calibrating the scores of biased reviewers of scientific papers

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q267651)