Tighter Confidence Intervals for Rating Systems

arXiv1912.03528MaRDI QIDQ6330664FDOQ6330664

Authors: Robert D. Nowak, Ervin Tánczos 

Publication date: 7 December 2019

Abstract: Rating systems are ubiquitous, with applications ranging from product recommendation to teaching evaluations. Confidence intervals for functionals of rating data such as empirical means or quantiles are critical to decision-making in various applications including recommendation/ranking algorithms. Confidence intervals derived from standard Hoeffding and Bernstein bounds can be quite loose, especially in small sample regimes, since these bounds do not exploit the geometric structure of the probability simplex. We propose a new approach to deriving confidence intervals that are tailored to the geometry associated with multi-star/value rating systems using a combination of techniques from information theory, including Kullback-Leibler, Sanov, and Csisz{'a}r inequalities. The new confidence intervals are almost always as good or better than all standard methods and are significantly tighter in many situations. The standard bounds can require several times more samples than our new bounds to achieve specified confidence interval widths.

This page was built for publication: Tighter Confidence Intervals for Rating Systems

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6330664)