Non‐word identification or spell checking without a dictionary

Donald C. Comeau and W. John Wilbur

Journal of the American Society for Information Science and Technology, 2004, vol. 55, issue 2, 169-177

Abstract: MEDLINE® is a collection of more than 12 million references and abstracts covering recent life science literature. With its continued growth and cutting‐edge terminology, spell‐checking with a traditional lexicon based approach requires significant additional manual follow‐up. In this work, an internal corpus based context quality rating α, frequency, and simple misspelling transformations are used to rank words from most likely to be misspellings to least likely. Eleven‐point average precisions of 0.891 have been achieved within a class of 42,340 all alphabetic words having an α score less than 10. Our models predict that 16,274 or 38% of these words are misspellings. Based on test data, this result has a recall of 79% and a precision of 86%. In other words, spell checking can be done by statistics instead of with a dictionary. As an application we examine the time history of low α words in MEDLINE® titles and abstracts.

Date: 2004
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1002/asi.10354

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:jamist:v:55:y:2004:i:2:p:169-177

Ordering information: This journal article can be ordered from
https://doi.org/10.1002/(ISSN)1532-2890

Access Statistics for this article

More articles in Journal of the American Society for Information Science and Technology from Association for Information Science & Technology
Bibliographic data for series maintained by Wiley Content Delivery ().