Attention-Based Multimodal Emotion Recognition for Fine-Grained Visual Ad Engagement Prediction on Instagram

Xin Lu and Zihan Li

Pinnacle Academic Press Proceedings Series, 2025, vol. 3, 204-218

Abstract: This paper presents a novel Attention-Based Multimodal Framework (ABMF) for emotion recognition and fine-grained engagement prediction in Instagram advertisements. Traditional approaches to advertisement assessment rely primarily on unimodal analysis and fail to capture the nuanced relationship between emotional content and engagement behaviors. The proposed framework integrates visual, textual, and metadata features through cross-modal attention mechanisms that dynamically identify emotionally salient components across modalities. We construct and annotate the Instagram Advertisement Emotion Dataset (IAED) containing 10,000 sponsored posts with valence-arousal ratings and engagement metrics. Experimental results demonstrate that ABMF achieves significant improvements over state-of-the-art baselines, with 12.1% reduction in valence MAE and 7.1% improvement in engagement prediction MAP. The research reveals distinct relationships between emotional dimensions and specific engagement behaviors: high arousal content generates 78.6% higher share rates while positive valence drives 62.7% more likes compared to negative content. The findings provide quantifiable insights for optimizing emotional content in advertisements based on campaign objectives. The cross-modal attention mechanism enables precise identification of engagement-driving features, offering Instagram advertisers a computational approach to predict and enhance user engagement through targeted emotional content design.

Keywords: multimodal emotion recognition; attention mechanisms; computational advertising; social media engagement prediction (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://pinnaclepubs.com/index.php/PAPPS/article/view/293/300 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:dba:pappsa:v:3:y:2025:i::p:204-218

Access Statistics for this article

More articles in Pinnacle Academic Press Proceedings Series from Pinnacle Academic Press
Bibliographic data for series maintained by Joseph Clark ().