The Influence of Status on Evaluations: Evidence from Online Coding Contests
Authors: Deodhar, Swanand J.; Babar, Yash; Burtch, Gordon
Journal: MIS Quarterly (2022)
<jats:p>In many instances, online contest platforms rely on contestants to ensure submission quality. This scalable evaluation mechanism offers a collective benefit. However, contestants may also leverage it to achieve personal, competitive benefits. Our study examines this tension from a status-theoretic perspective, suggesting that the conflict between competitive and collective benefits, and the net implication for evaluation efficacy, is influenced by contestants’ status. On the one hand, contestants of lower status may be viewed as less skilled and hence more likely to make mistakes. Therefore, low-status contestants may attract more evaluations if said evaluations are driven predominantly by an interest in collective benefits. On the other hand, if evaluations are driven largely by an interest in personal, competitive benefits, a low-status contestant makes for a less attractive target and hence may attract fewer evaluations. We empirically test these competing possibilities using a dataset of coding contests from Codeforces. The platform allows contestants to assess others’ submissions and improve evaluations (a collective benefit) by devising test cases (hacks) in addition…