An experiment in index term frequency

Elaine Svenonius

Journal of the American Society for Information Science, 1972, vol. 23, issue 2, 109-121

Abstract: This paper presents an experimental study of index‐term frequency as a factor in retrieval performance. The frequency of an index term, or its “breadth” as it is called here, is the number of postings made to the term in a given collection. The question is asked: Of index terms assigned to documents, which function most effectively in retrieval, the most used or popular terms, or those which are used relatively infrequently? The experiment is a retrieval experiment and uses the Cran‐field‐Salton data. Breadth of indexing is varied by nonrandomly deleting terms from documents. Retrieval output is evaluated using the Expected Search Length measure of retrieval effectiveness as well as the usual precision and recall. The Wilcoxen Test is used to determine the statistical significance of the different indexings. The results show that the “optimal” breadth of indexing is a variable, depending on user needs: if a few documents are wanted or high precision is desired, then narrow terms are more effective than broad ones; if, on the other hand, all or most relevant documents are wanted, then broad terms are better. An argument, however, can be made for the quality of narrow terms, since when these terms are deleted precision never improves, whereas deleting broad terms always results in higher precision. A corollary experiment is carried out to compare two indexings of the same average breadth where one indexing consists of semantically appropriate terms—terms taken from the document title—and the other consists of merely “reasonable” index terms. The results suggest that title‐term indexing is qualifiedly superior.

Date: 1972
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1002/asi.4630230205

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:jamest:v:23:y:1972:i:2:p:109-121

Ordering information: This journal article can be ordered from
https://doi.org/10.1002/(ISSN)1097-4571

Access Statistics for this article

More articles in Journal of the American Society for Information Science from Association for Information Science & Technology
Bibliographic data for series maintained by Wiley Content Delivery ().