Novelty detection for topic tracking

Cem Aksoy, Fazli Can and Seyit Kocberber

Journal of the American Society for Information Science and Technology, 2012, vol. 63, issue 4, 777-795

Abstract: Multisource web news portals provide various advantages such as richness in news content and an opportunity to follow developments from different perspectives. However, in such environments, news variety and quantity can have an overwhelming effect. New‐event detection and topic‐tracking studies address this problem. They examine news streams and organize stories according to their events; however, several tracking stories of an event/topic may contain no new information (i.e., no novelty). We study the novelty detection (ND) problem on the tracking news of a particular topic. For this purpose, we build a Turkish ND test collection called BilNov‐2005 and propose the usage of three ND methods: a cosine‐similarity (CS)‐based method, a language‐model (LM)‐based method, and a cover‐coefficient (CC)‐based method. For the LM‐based ND method, we show that a simpler smoothing approach, Dirichlet smoothing, can have similar performance to a more complex smoothing approach, Shrinkage smoothing. We introduce a baseline that shows the performance of a system with random novelty decisions. In addition, a category‐based threshold learning method is used for the first time in ND literature. The experimental results show that the LM‐based ND method significantly outperforms the CS‐ and CC‐based methods, and category‐based threshold learning achieves promising results when compared to general threshold learning.

Date: 2012
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1002/asi.21697

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:jamist:v:63:y:2012:i:4:p:777-795

Ordering information: This journal article can be ordered from
https://doi.org/10.1002/(ISSN)1532-2890

Access Statistics for this article

More articles in Journal of the American Society for Information Science and Technology from Association for Information Science & Technology
Bibliographic data for series maintained by Wiley Content Delivery ().