Transfer Learning for Multi-Premise Entailment with Relationship Processing Module

Pin Wu, Rukang Zhu and Zhidan Lei
Additional contact information
Pin Wu: School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China
Rukang Zhu: School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China
Zhidan Lei: School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China

Future Internet, 2021, vol. 13, issue 3, 1-13

Abstract: Using the single premise entailment (SPE) model to accomplish the multi-premise entailment (MPE) task can alleviate the problem that the neural network cannot be effectively trained due to the lack of labeled multi-premise training data. Moreover, the abundant judgment methods for the relationship between sentence pairs can also be applied in this task. However, the single-premise pre-trained model does not have a structure for processing multi-premise relationships, and this structure is a crucial technique for solving MPE problems. This paper proposes adding a multi-premise relationship processing module based on not changing the structure of the pre-trained model to compensate for this deficiency. Moreover, we proposed a three-step training method combining this module, which ensures that the module focuses on dealing with the multi-premise relationship during matching, thus applying the single-premise model to multi-premise tasks. Besides, this paper also proposes a specific structure of the relationship processing module, i.e., we call it the attention-backtracking mechanism. Experiments show that this structure can fully consider the context of multi-premise, and the structure combined with the three-step training can achieve better accuracy on the MPE test set than other transfer methods.

Keywords: transfer learning; multi-premise entailment; natural language inference; attention mechanism (search for similar items in EconPapers)
JEL-codes: O3 (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1999-5903/13/3/71/pdf (application/pdf)
https://www.mdpi.com/1999-5903/13/3/71/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jftint:v:13:y:2021:i:3:p:71-:d:516337

Access Statistics for this article

Future Internet is currently edited by Ms. Grace You

More articles in Future Internet from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().