Criteria for performance evaluation

David J. Weiss, Kristin Brennan, Rick Thomas, Alex Kirlik and Sarah M. Miller

Judgment and Decision Making, 2009, vol. 4, issue 2, pages 164-174

Abstract: Using a cognitive task (mental calculation) and a perceptual-motor task (stylized golf putting), we examined differential proficiency using the CWS index and several other quantitative measures of performance. The CWS index (Weiss \& Shanteau, 2003) is a coherence criterion that looks only at internal properties of the data without incorporating an external standard. In Experiment 1, college students (n = 20) carried out 2- and 3-digit addition and multiplication problems under time pressure. In Experiment 2, experienced golfers (n = 12), also college students, putted toward a target from nine different locations. Within each experiment, we analyzed the same responses using different methods. For the arithmetic tasks, accuracy information (mean absolute deviation from the correct answer, MAD) using a coherence criterion was available; for golf, accuracy information using a correspondence criterion (mean deviation from the target, also MAD) was available. We ranked the performances of the participants according to each measure, then compared the orders using Spearman's r\textsubscript{s}. For mental calculation, the CWS order correlated moderately (r\textsubscript{s} =.46) with that of MAD. However, a different coherence criterion, degree of model fit, did not correlate with either CWS or accuracy. For putting, the ranking generated by CWS correlated .68 with that generated by MAD. Consensual answers were also available for both experiments, and the rankings they generated correlated highly with those of MAD. The coherence vs.\ correspondence distinction did not map well onto criteria for performance evaluation.

Keywords: arithmetic; CWS index; judgment; measurement. (search for similar items in EconPapers)
Date: 2009
View list of references

Downloads: (external link)
http://journal.sjdm.org/ccw/ccw.pdf (application/pdf)
http://journal.sjdm.org/ccw/ccw.html (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: http://EconPapers.repec.org/RePEc:jdm:journl:v:4:y:2009:i:2:p:164-174

Access Statistics for this article

Judgment and Decision Making is edited by Jonathan Baron

More articles in Judgment and Decision Making from Society for Judgment and Decision Making
Series data maintained by Jonathan Baron ().