Order effect in relevance judgment

Yunjie Xu and Dong Wang

Journal of the American Society for Information Science and Technology, 2008, vol. 59, issue 8, 1264-1275

Abstract: The order effect of relevance judgment refers to the different relevance perceptions of a document when it appears in different positions in a list. Although the order effect of relevance judgment has significant theoretical and practical implications, the extant literature is inconclusive regarding its existence and forming mechanisms. This study proposes a set of order effect forming mechanisms, including the learning effect, the subneed scheduling effect, and the cursoriness effect based on the conceptualization of dynamic relevance and the psychology of cognitive elaboration. Our empirical study indicates that in an interactive information retrieval setting, when a document list is reasonably long, order effects demonstrate a curvilinear pattern that conforms to the combined effect of the three mechanisms. Moreover, the curvilinear pattern of order effect could differ for documents of different relevance levels.

Date: 2008
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://doi.org/10.1002/asi.20826

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:jamist:v:59:y:2008:i:8:p:1264-1275

Ordering information: This journal article can be ordered from
https://doi.org/10.1002/(ISSN)1532-2890

Access Statistics for this article

More articles in Journal of the American Society for Information Science and Technology from Association for Information Science & Technology
Bibliographic data for series maintained by Wiley Content Delivery ().