An Enhanced Term Weighted Question Embedding for Visual Question Answering

Sruthy Manmadhan and Binsu C Kovoor ()
Additional contact information
Sruthy Manmadhan: Division of Information Technology, Cochin University of Science and Technology, Kerala, 682022, India†Department of CSE, NSS College of Engineering, Kerala, India
Binsu C Kovoor: Division of Information Technology, Cochin University of Science and Technology, Kerala, 682022, India

Journal of Information & Knowledge Management (JIKM), 2022, vol. 21, issue 02, 1-23

Abstract: Visual Question Answering (VQA) is a multi-modal AI-complete task of answering natural language questions about images. Literature solved VQA with a three-phase pipeline: image and question featurisation, multi-modal feature fusion and answer generation or prediction. Most of the works have given attention to the second phase, where multi-modal features get combined ignoring the effect of individual input features. This work investigates VQA’s natural language question embedding phase by proposing a new question featurisation framework based on Supervised Term Weighting (STW) schemes. In addition, two new STW schemes integrating text semantics, qf.cos and tf.rf.sim, have been introduced to boost the framework’s performance. A series of tests on the DAQUAR VQA dataset is used to compare the new system to conventional pre-trained word embedding. Over the past few years, STW schemes have been commonly used in text classification research. In light of this, tests are carried out to verify the effectiveness of the two newly proposed STW schemes in the general text classification task.

Keywords: Text classification; semantic similarity; supervised term weighting; visual question answering (search for similar items in EconPapers)
Date: 2022
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.worldscientific.com/doi/abs/10.1142/S0219649222500289
Access to full text is restricted to subscribers

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wsi:jikmxx:v:21:y:2022:i:02:n:s0219649222500289

Ordering information: This journal article can be ordered from

DOI: 10.1142/S0219649222500289

Access Statistics for this article

Journal of Information & Knowledge Management (JIKM) is currently edited by Professor Suliman Hawamdeh

More articles in Journal of Information & Knowledge Management (JIKM) from World Scientific Publishing Co. Pte. Ltd.
Bibliographic data for series maintained by Tai Tone Lim ().