Feature Fusion Text Classification Model Combining CNN and BiGRU with Multi-Attention Mechanism

Jingren Zhang, Fang’ai Liu, Weizhi Xu and Hui Yu
Additional contact information
Jingren Zhang: School of Information Science and Engineering, Shandong Normal University, Jinan 250358, China
Fang’ai Liu: School of Information Science and Engineering, Shandong Normal University, Jinan 250358, China
Weizhi Xu: School of Information Science and Engineering, Shandong Normal University, Jinan 250358, China
Hui Yu: School of Business, Shandong Normal University, Jinan 250358, China

Future Internet, 2019, vol. 11, issue 11, 1-24

Abstract: Convolutional neural networks (CNN) and long short-term memory (LSTM) have gained wide recognition in the field of natural language processing. However, due to the pre- and post-dependence of natural language structure, relying solely on CNN to implement text categorization will ignore the contextual meaning of words and bidirectional long short-term memory (BiLSTM). The feature fusion model is divided into a multiple attention (MATT) CNN model and a bi-directional gated recurrent unit (BiGRU) model. The CNN model inputs the word vector (word vector attention, part of speech attention, position attention) that has been labeled by the attention mechanism into our multi-attention mechanism CNN model. Obtaining the influence intensity of the target keyword on the sentiment polarity of the sentence, and forming the first dimension of the sentiment classification, the BiGRU model replaces the original BiLSTM and extracts the global semantic features of the sentence level to form the second dimension of sentiment classification. Then, using PCA to reduce the dimension of the two-dimensional fusion vector, we finally obtain a classification result combining two dimensions of keywords and sentences. The experimental results show that the proposed MATT-CNN+BiGRU fusion model has 5.94% and 11.01% higher classification accuracy on the MRD and SemEval2016 datasets, respectively, than the mainstream CNN+BiLSTM method.

Keywords: BiGRU; multi-attention; MATT-CNN+BiGRU; PCA (search for similar items in EconPapers)
JEL-codes: O3 (search for similar items in EconPapers)
Date: 2019
References: View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.mdpi.com/1999-5903/11/11/237/pdf (application/pdf)
https://www.mdpi.com/1999-5903/11/11/237/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jftint:v:11:y:2019:i:11:p:237-:d:286141

Access Statistics for this article

Future Internet is currently edited by Ms. Grace You

More articles in Future Internet from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().