Hierarchical Hybrid Neural Networks With Multi-Head Attention for Document Classification

Weihao Huang, Jiaojiao Chen, Qianhua Cai, Xuejie Liu, Yudong Zhang and Xiaohui Hu
Additional contact information
Weihao Huang: School of Electronics and Information Engineering, School of Physics and Telecommunication Engineering, South China Normal University, China
Jiaojiao Chen: School of Electronics and Information Engineering, South China Normal University, China
Qianhua Cai: School of Electronics and Information Engineering, South China Normal University, China
Xuejie Liu: School of Electronics and Information Engineering, South China Normal University, China
Yudong Zhang: School of Informatics, University of Leicester, UK
Xiaohui Hu: School of Electronics and Information Engineering, South China Normal University, China

International Journal of Data Warehousing and Mining (IJDWM), 2022, vol. 18, issue 1, 1-16

Abstract: Document classification is a research topic aiming to predict the overall text sentiment polarity with the advent of deep neural networks. Various deep learning algorithms have been employed in the current studies to improve classification performance. To this end, this paper proposes a hierarchical hybrid neural network with multi-head attention (HHNN-MHA) model on the task of document classification. The proposed model contains two layers to deal with the word-sentence level and sentence-document level classification respectively. In the first layer, CNN is integrated into Bi-GRU and a multi-head attention mechanism is employed, in order to exploit local and global features. Then, both Bi-GRU and attention mechanism are applied to document processing and classification in the second layer. Experiments on four datasets demonstrate the effectiveness of the proposed method. Compared to the state-of-art methods, our model achieves competitive results in document classification in terms of experimental performance.

Date: 2022
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://services.igi-global.com/resolvedoi/resolve.aspx?doi=10.4018/IJDWM.303673 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:igg:jdwm00:v:18:y:2022:i:1:p:1-16

Access Statistics for this article

International Journal of Data Warehousing and Mining (IJDWM) is currently edited by Eric Pardede

More articles in International Journal of Data Warehousing and Mining (IJDWM) from IGI Global
Bibliographic data for series maintained by Journal Editor ().