An image and text-based fake news detection with transfer learning

Esther Irawati Setiawan, Patrick Sutanto, Christian Nathaniel Purwanto, Joan Santoso, Ferdinandus Fx, Nemuel Daniel Pah and Mauridhi Hery Purnomo

PLOS ONE, 2025, vol. 20, issue 6, 1-12

Abstract: Fake news has emerged as a significant problem in today’s information age, threatening the reliability of information sources. Detecting fake news is crucial for maintaining trust and ensuring access to factual information. While deep learning offers solutions, most approaches focus on the text, neglecting the potential of visual information, which may contradict or misrepresent the accompanying text. This research proposes a multimodal classification approach that combines text and images to improve fake news detection, particularly in low-resource settings where labeled data is scarce. We leverage CLIP, a model that understands relationships between images and text, to extract features from both modalities. These features are concatenated and fed into a simple one-layer multi-layer perceptron (MLP) for classification. To enhance data efficiency, we apply LoRA (Low-Rank Adaptation), a parameter-efficient fine-tuning technique, to the CLIP model. We also explore the effects of integrating features from other models. The model achieves an 83% accuracy when using LoRA in classifying whether an image and its accompanying text constitute fake or factual news. These results highlight the potential of multimodal learning and efficient fine-tuning techniques for robust fake news detection, even with limited data.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0324394 (text/html)
https://journals.plos.org/plosone/article/file?id= ... 24394&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pone00:0324394

DOI: 10.1371/journal.pone.0324394

Access Statistics for this article

More articles in PLOS ONE from Public Library of Science
Bibliographic data for series maintained by plosone ().