Computer selection of keywords using word‐frequency analysis

John M. Carroll and Robert Roeloffs

American Documentation, 1969, vol. 20, issue 3, 227-233

Abstract: A statistically based method for automatically identifying keywords in machine‐readable text has been developed which produces keyword lists that agree better with composite lists produced by panels of human indexers than do lists produced by five statistical criteria previously suggested, and also better than lists produced by most of the individual panel members. The method makes use of both the in‐document word‐occurrence frequency and the in‐corpus relative occurrence frequency as measures of word importance. Each statistical criterion was compared with the performance of human indexers by the use of rank correlation statistics. The simple word count was found to be superior to the other four previously suggested criteria—all of which made use of the in‐corpus relative occurrence frequency. The tests were conducted over 19 documents dealing with the subject of Information Science, a total of over 66,000 word occurrences. Seventeen indexers representing eight different information centers participated in experiments.

Date: 1969
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://doi.org/10.1002/asi.4630200308

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:amedoc:v:20:y:1969:i:3:p:227-233

Ordering information: This journal article can be ordered from
https://doi.org/10.1002/(ISSN)1936-6108

Access Statistics for this article

American Documentation is currently edited by Javed Mostafa

More articles in American Documentation from Wiley Blackwell
Bibliographic data for series maintained by Wiley Content Delivery ().