Employing multiple representations for Chinese information retrieval

K.L. Kwok

Journal of the American Society for Information Science, 1999, vol. 50, issue 8, 709-723

Abstract: For information retrieval in the Chinese language, three representation methods for texts are popular, namely: 1‐gram or character, bigram, and short‐word. Each has its advantages as well as drawbacks. Employing more than one method may combine advantages from them and enhance retrieval effectiveness. We investigated two ways of using them simultaneously: mixing representations in documents and queries, and combining retrieval lists obtained via different representations. The experiments were done with the 170 MB evaluated Chinese corpora and 54 long and short queries available from the (TREC) program and using our Probabilistic Indexing and Retrieval Components System (PIRCS retrieval system). Experiments show that good retrieval need not depend on accurate word segmentation; approximate segmentation into short‐words will do. Results also show and confirm that bigram representation alone works well; mixing characters with bigram representation boosts effectiveness further, but it is preferable to mix characters with short‐word indexing which is more efficient, needs less resource, and gives better retrieval more often. Combining retrieval lists from short‐word with character representation and from bigram indexing provides the best retrieval results but also at a substantial cost.

Date: 1999
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1002/(SICI)1097-4571(1999)50:83.0.CO;2-V

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:jamest:v:50:y:1999:i:8:p:709-723

Ordering information: This journal article can be ordered from
https://doi.org/10.1002/(ISSN)1097-4571

Access Statistics for this article

More articles in Journal of the American Society for Information Science from Association for Information Science & Technology
Bibliographic data for series maintained by Wiley Content Delivery ().