Object-Assisted Question Featurization and Multi-CNN Image Feature Fusion for Visual Question Answering

Sruthy Manmadhan and Binsu C. Kovoor
Additional contact information
Sruthy Manmadhan: School of Engineering, Cochin University of Science and Technology, India & Department of Computer Science and Engineering, NSS College of Engineering, India
Binsu C. Kovoor: School of Engineering, Cochin University of Science and Technology, India

International Journal of Intelligent Information Technologies (IJIIT), 2023, vol. 19, issue 1, 1-19

Abstract: Visual question answering (VQA) demands a meticulous and concurrent proficiency in image interpretation and natural language understanding to correctly answer the question about an image. The existing VQA solutions either focus only on improving the joint multi-modal embedding or on the fine-tuning of visual understanding through attention. This research, in contrast to the current trend, investigates the feasibility of an object-assisted language understanding strategy titled semantic object ranking (SOR) framework for VQA. The proposed system refines the natural language question representation with the help of detected visual objects. For multi-CNN image representation, the system employs canonical correlation analysis (CCA). The suggested model is assessed using accuracy and WUPS measures on the DAQUAR dataset. On the DAQUAR dataset, the analytical outcomes reveal that the presented system outperforms the prior state-of-the-art by a significant factor. In addition to the quantitative analysis, proper illustrations are supplied to observe the reasons for performance improvement.

Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://services.igi-global.com/resolvedoi/resolve.aspx?doi=10.4018/IJIIT.318671 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:igg:jiit00:v:19:y:2023:i:1:p:1-19

Access Statistics for this article

International Journal of Intelligent Information Technologies (IJIIT) is currently edited by Vijayan Sugumaran

More articles in International Journal of Intelligent Information Technologies (IJIIT) from IGI Global
Bibliographic data for series maintained by Journal Editor ().