Extreme Sports Evaluation: Evidence from Judging Figure Skating

Jungmin Lee

No 122, Econometric Society 2004 North American Summer Meetings from Econometric Society

Abstract: The quality of subjective performance evaluation is dependent on the incentive structures faced by evaluators, in particular on how they are monitored and themselves evaluated. Figure skating competitions provide a unique opportunity to study subjective evaluation. This paper develops a simple model of "outlier aversion bias" in which evaluators avoid submitting outlying judgments, because they are later assessed by the degree to which their scores match those of the other judges. I employ two empirical strategies: first, individual judges are traced, and within-individual dynamic correlations of scores are tested for by using the standard dynamic panel data model. Second, I exploit a natural experiment in which anonymity and random selection of judges have been introduced under new judging system in 2003. I find significant evidence for the existence of outlier aversion bias. Individual judges manipulate scores to achieve a targeted level of agreement with the other judges, and the dispersion of scores depends upon the type of judge-assessment system. One major implication of this paper, which is at variance with the industrial psychology and personnel management literature, is that agreement may not be a good criterion for the validity of an evaluation system. The price of agreement is at least a 14 percent loss of information

Keywords: Subjective Evaluation; Rational Herding (search for similar items in EconPapers)
JEL-codes: D7 M5 (search for similar items in EconPapers)
Date: 2004-08-11
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ecm:nasm04:122

Access Statistics for this paper

More papers in Econometric Society 2004 North American Summer Meetings from Econometric Society Contact information at EDIRC.
Bibliographic data for series maintained by Christopher F. Baum (baum@bc.edu).