Dynamic paired-comparison scaling.

DOI10.1016/0022-2496(92)90036-7MaRDI QIDQ1193376zbMATH OpenOpenAlexFDO

Authors William H. Batchelder, N. J. Bershad, R. Simpson

Publication date 27 September 1992

Published in Journal of Mathematical Psychology (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1016/0022-2496(92)90036-7

Paired and multiple comparisons; multiple testing (62J15) Sequential estimation (62L12) Applications of statistics to psychology (62P15) Monte Carlo methods (65C05)

Review text

The problem of paired-comparison scaling when underlying scale values may change from trail to trail is studied. The Elo rating system used throughout the world for rating chess ability is presented as an analogy. The rational procedures to provide sequentially revised estimates of the reward function is proposed. The reward system is defined based on the correspondent values of the difference between current scale estimates called ratings and binary choice data. The formal definitions of the sequential paired-comparison system as well as correspondent concepts of monotone paired-comparison system, uniform system and dynamic monotone paired-comparison system are provided. The sequential paired-comparison (estimation) scheme is described to handle the case where underlying ratings may not be stationary. The set of axioms (Zero Sum, Fair Game, Constant Value and Constant Variance) is given for reward system that relate with each other reward, loss and cumulative distribution functions. Certain observations about reward systems satisfied the axioms for sequential paired-comparison are described. Asymptotic properties of the reward system with a certain condition on a distribution function are discussed. The authors propose the estimation theory for sequential paired-comparison with particular examples for two objects, one object and many objects estimation. It is shown that the Elo chess rating system was based on the special case of the presented theory (so called uniform model). The interpretation of the Bush-Mosteller linear operator learning model as two-object estimation problem with a uniform reward system is described. The results obtained can be used to determine the approximate behaviour of reward systems under conditions of fixed rating. The results of linear and diffusion approximation methods and Monte Carlo Simulation are provided for the particular examples. One-object estimation approach is proved to be formally equivalent to some methods of the item response theory and sequential models in psychophysical threshold determination. The results obtained are presented as relations between the conditional expected ratings, current estimates and true ratings of the objects on some trail. They provide a generalisation to be applied to adaptive estimation procedures used in psychophysical and mental testing in the cases where an underlying ability of the test object or other parameters of the experiment might change from trial to trial. The estimation theory for many objects is shown as a sequential estimation model for dynamic paired-comparison system. This case is studied with respect to the introduced axioms. The problem of sequential estimation with a ``draw value or no-preference as an allowed estimation result is stated. The Axiom (Draw is Half a Win) is introduced for this case. The estimation theory for two objects, one object and many objects estimation for this case is left for further investigation.

Recommendations

Cites work

Cited in

(12)

This page was built for publication: Dynamic paired-comparison scaling.

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1193376)