Multi-modal recommendation algorithm fusing visual and textual features

Xuefeng Hu, Wenting Yu, Yun Wu and Yukang Chen

PLOS ONE, 2023, vol. 18, issue 6, 1-18

Abstract: In recommender systems, the lack of interaction data between users and items tends to lead to the problem of data sparsity and cold starts. Recently, the interest modeling frameworks incorporating multi-modal features are widely used in recommendation algorithms. These algorithms use image features and text features to extend the available information, which alleviate the data sparsity problem effectively, but they also have some limitations. On the one hand, multi-modal features of user interaction sequences are not considered in the interest modeling process. On the other hand, the aggregation of multi-modal features often employs simple aggregators, such as sums and concatenation, which do not distinguish the importance of different feature interactions. In this paper, to tackle this, we propose the FVTF (Fusing Visual and Textual Features) algorithm. First, we design a user history visual preference extraction module based on the Query-Key-Value attention to model users’ historical interests by using of visual features. Second, we design a feature fusion and interaction module based on the multi-head bit-wise attention to adaptively mine important feature combinations and update the higher-order attention fusion representation of features. We conduct experiments on the Movielens-1M dataset, and the experiments show that FVTF achieved the best performance compared with the benchmark recommendation algorithms.

Date: 2023
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0287927 (text/html)
https://journals.plos.org/plosone/article/file?id= ... 87927&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pone00:0287927

DOI: 10.1371/journal.pone.0287927

Access Statistics for this article

More articles in PLOS ONE from Public Library of Science
Bibliographic data for series maintained by plosone ().