Design and analysis of the NIPS 2016 review process

MaRDI QIDQ4558199zbMATH OpenFDO

Authors Nihar B. Shah, Behzad Tabibian, Krikamol Muandet, Isabelle Guyon, Ulrike Von Luxburg

Publication date 21 November 2018

Full work available at URL https://arxiv.org/abs/1708.09794, http://jmlr.csail.mit.edu/papers/v19/17-511.html

zbMATH Keywords

consistencyNeural Information Processing System (NIPS)ordinalpeer reviewpost hoc analysis

Mathematics Subject Classification ID

Applications of statistics to social sciences (62P25) Learning and adaptive systems in artificial intelligence (68T05) Informational aspects of data analysis and big data (94A16)

Recommendations

• Advances in neural information processing systems 19. Proceedings of the 2006 conference, Vancouver, BC, Canada, December 4--6, 2006
• Advances in neural information processing 18. Proceedings of the 2005 conference, Vancouver, Canada, December 2005.
• Advances in neural information processing systems 17. Proceedings of the 2004 conference, Vancouver, BC, Canada, December 13--16, 2004.
• Assigning papers to referees
• A statistical approach to calibrating the scores of biased reviewers of scientific papers

Cites work

• Consensus and Cooperation in Networked Multi-Agent Systems
• Estimation from pairwise comparisons: sharp minimax bounds with topology dependence
• Stochastically Transitive Models for Pairwise Comparisons: Statistical and Computational Issues
• Uncertain Judgements: Eliciting Experts' Probabilities

Cited in

(8)

• Quantitative measurements of the influence of participant roles during peer review meetings
• A statistical approach to calibrating the scores of biased reviewers of scientific papers
• scientific article; zbMATH DE number 7415106 (Why is no real title available?)
• Advances in neural information processing systems 19. Proceedings of the 2006 conference, Vancouver, BC, Canada, December 4--6, 2006
• \textsc{PeerNomination}: a novel peer selection algorithm to handle strategic and noisy assessments
• Modeling Preferences: A Bayesian Mixture of Finite Mixtures for Rankings and Ratings
• Debiasing evaluations that are biased by evaluations
• When zero may not be zero: a cautionary note on the use of inter-rater reliability in evaluating grant peer review

This page was built for publication: Design and analysis of the NIPS 2016 review process

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4558199)