A type‐token identity in the Simon‐Yule model of text

Ye‐Sho Chen and Ferdinand F. Leimkuhler

Journal of the American Society for Information Science, 1989, vol. 40, issue 1, 45-53

Abstract: There are three significant results in this paper. First, we establish a type‐token identity relating the type‐token ratio and the bilogarithmic type‐token ratio. The plays of Shakespeare and other interesting texts serve as demonstrative examples. Second, the Simon‐Yule model of Zipf's law is used to derive the type‐token identity and provide a promising statistical model of text generation. Third, a realistic refinement of the Simon‐Yule model is made to allow for a decreasing entry rate of new words. Simulation methods are used to show that the type‐token identity is preserved with this change in assumptions. © 1989 John Wiley & Sons, Inc.

Date: 1989
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1002/(SICI)1097-4571(198901)40:13.0.CO;2-S

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:jamest:v:40:y:1989:i:1:p:45-53

Ordering information: This journal article can be ordered from
https://doi.org/10.1002/(ISSN)1097-4571

Access Statistics for this article

More articles in Journal of the American Society for Information Science from Association for Information Science & Technology
Bibliographic data for series maintained by Wiley Content Delivery ().