Measuring retrieval effectiveness based on user preference of documents

Y. Y. Yao

Journal of the American Society for Information Science, 1995, vol. 46, issue 2, 133-145

Abstract: The notion of user preference is adopted for the representation, interpretation, and measurement of the relevance or usefulness of documents. User judgments on documents may be formally described by a weak order (i.e., user ranking) and measured using an ordinal scale. Within this framework, a new measure of system performance is suggested based on the distance between user ranking and system ranking. It only uses the relative order of documents and therefore confirms to the valid use of an ordinal scale measuring relevance. It is also applicable to multilevel relevance judgments and ranked system output. The appropriateness of the proposed measure is demonstrated through an axiomatic approach. The inherent relationships between the new measure and many existing measures provide further supporting evidence. © 1995 John Wiley & Sons, Inc.

Date: 1995
References: Add references at CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
https://doi.org/10.1002/(SICI)1097-4571(199503)46:23.0.CO;2-Z

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:jamest:v:46:y:1995:i:2:p:133-145

Ordering information: This journal article can be ordered from
https://doi.org/10.1002/(ISSN)1097-4571

Access Statistics for this article

More articles in Journal of the American Society for Information Science from Association for Information Science & Technology
Bibliographic data for series maintained by Wiley Content Delivery ().