A comparison of group and individual performance among subject experts and untrained workers at the document retrieval task

W. John Wilbur

Journal of the American Society for Information Science, 1998, vol. 49, issue 6, 517-529

Abstract: Useful retrieval depends on the ability to predict which documents a user will find helpful in answer to a query. Our interest is the common case when no information is provided about the user other than the query and the query is in natural language. In this setting it is well accepted that a human can make useful predictions in the form of judgments about what will likely prove useful to another human. We present data showing that when the predictions of a group of humans are averaged, the result is a better predictor. If performance is measured as a precision, the group performance increases with the size of the group and approaches a limit of approximately 50% improvement over average individual performance on our data. Superior performance by groups raises the question of how. The groups we studied were subject experts and a natural question was whether the superior performance resulted from the pooling of their subject knowledge. In order to answer this question we studied also a group of untrained individuals. To our surprise we found that while untrained individuals had a somewhat inferior performance compared to trained individuals, the group of untrained individuals together performed better than any single trained individual and almost at the level of the trained group. © 1998 John Wiley & Sons, Inc.

Date: 1998
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://doi.org/10.1002/(SICI)1097-4571(19980501)49:63.0.CO;2-T

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:jamest:v:49:y:1998:i:6:p:517-529

Ordering information: This journal article can be ordered from
https://doi.org/10.1002/(ISSN)1097-4571

Access Statistics for this article

More articles in Journal of the American Society for Information Science from Association for Information Science & Technology
Bibliographic data for series maintained by Wiley Content Delivery ().