JGRCAN: A Visual Question Answering Co-Attention Network via Joint Grid-Region Features

Jianpeng Liang, Tianjiao Xu, Shihong Chen, Zhuopan Ao and Yong Zhang

Mathematical Problems in Engineering, 2022, vol. 2022, 1-11

Abstract: In recent years, region features extracted from target detection networks have played an important role in visual question answering. The region features only extract the areas that are related to the target, but they lose a lot of nontarget context information and fine-grained details. On the contrary, the grid feature does not lose the details of nontargets but is not conducive to the recognition of the counting question of multiple small targets in the image. To solve this problem, this paper proposes a visual question answering network via joint grid-region features (JGRCAN), which consists of a feature extraction layer, co-attention layer, and fusion layer. The feature extraction layer includes extracting grid features and region features from the image and text features from the question and extracting multivisual feature representation and question feature representation through the co-attention layer to output attention weight and attention feature representation, respectively. The proposed approach effectively integrates grid features and region features, realizes the complementary advantages of region features and grid features, and is able to accurately focus on areas of the image that are relevant to the answer to the question. The results show that the overall classification accuracy of the algorithm on the test-dev and test-std subsets of VQA-v2 is 70.87% and 71.18%, respectively. Compared with baseline models, our proposed JGRCAN has good performance.

Date: 2022
References: Add references at CitEc
Citations:

Downloads: (external link)
http://downloads.hindawi.com/journals/mpe/2022/4554074.pdf (application/pdf)
http://downloads.hindawi.com/journals/mpe/2022/4554074.xml (application/xml)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:hin:jnlmpe:4554074

DOI: 10.1155/2022/4554074

Access Statistics for this article

More articles in Mathematical Problems in Engineering from Hindawi
Bibliographic data for series maintained by Mohamed Abdelhakeem ().