Building Multi-Modal Relational Graphs for Multimedia Retrieval

Jyh-Ren Shieh, Ching-Yung Lin, Shun-Xuan Wang and Ja-Ling Wu
Additional contact information
Jyh-Ren Shieh: National Taiwan University, Taiwan
Ching-Yung Lin: IBM T. J. Watson Research Center, USA
Shun-Xuan Wang: National Taiwan University, Taiwan
Ja-Ling Wu: National Taiwan University, Taiwan

International Journal of Multimedia Data Engineering and Management (IJMDEM), 2011, vol. 2, issue 2, 19-41

Abstract: The abundance of Web 2.0 social media in various media formats calls for integration that takes into account tags associated with these resources. The authors present a new approach to multi-modal media search, based on novel related-tag graphs, in which a query is a resource in one modality, such as an image, and the results are semantically similar resources in various modalities, for instance text and video. Thus the use of resource tagging enables the use of multi-modal results and multi-modal queries, a marked departure from the traditional text-based search paradigm. Tag relation graphs are built based on multi-partite networks of existing Web 2.0 social media such as Flickr and Wikipedia. These multi-partite linkage networks (contributor-tag, tag-category, and tag-tag) are extracted from Wikipedia to construct relational tag graphs. In fusing these networks, the authors propose incorporating contributor-category networks to model contributor’s specialization; it is shown that this step significantly enhances the accuracy of the inferred relatedness of the term-semantic graphs. Experiments based on 200 TREC-5 ad-hoc topics show that the algorithms outperform existing approaches. In addition, user studies demonstrate the superiority of this visualization system and its usefulness in the real world.

Date: 2011
References: Add references at CitEc
Citations:

Downloads: (external link)
http://services.igi-global.com/resolvedoi/resolve. ... 018/jmdem.2011040102 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:igg:jmdem0:v:2:y:2011:i:2:p:19-41

Access Statistics for this article

International Journal of Multimedia Data Engineering and Management (IJMDEM) is currently edited by Chengcui Zhang

More articles in International Journal of Multimedia Data Engineering and Management (IJMDEM) from IGI Global
Bibliographic data for series maintained by Journal Editor ().