An Enhanced Phrase Matching Method Based on Cross-Attention

Guoqing Pang, Qiming Fu, Jianping Chen, Yunzhe Wang, You Lu, Hongjie Wu and Vimal Shanmuganathan

Mathematical Problems in Engineering, 2023, vol. 2023, 1-11

Abstract: Text matching is a core problem in the field of natural language understanding. It aims to analyze and judge the semantic relevance or similarity between two texts. In the past, much work on text matching focused on long text and little on phrase optimization. However, phrase matching also has essential application scenarios. Compared with long texts, phrases have difficulties covering less semantics and polysemy, and because of the weak expression ability of phrases, it is hard to match phrases accurately. On the Kaggle patent phrase matching dataset, due to the few words in the phrase and the repeated occurrences under different patent classification numbers, it is difficult to match each other accurately. In the data-processing stage, this work proposes aggregating related targets for data fusion, expanding the background semantic information, and enhancing the expression ability of phrases. In the training stage, this work considers adding a cross-attention network to the model to make the additional related targets better used and learned. The added cross-attention network makes the model pay more attention to the most related information. The proposed method has experimented on pretrained language models such as BERT-for-patent, DeBERTa, and RoBERTa. The results show that the proposed method in this work improves by 2–4 points more than the general method without any information or additional network layers in its evaluation metric called the Pearson correlation, which enhances the performance in short-text matching.

Date: 2023
References: Add references at CitEc
Citations:

Downloads: (external link)
http://downloads.hindawi.com/journals/mpe/2023/3327065.pdf (application/pdf)
http://downloads.hindawi.com/journals/mpe/2023/3327065.xml (application/xml)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:hin:jnlmpe:3327065

DOI: 10.1155/2023/3327065

Access Statistics for this article

More articles in Mathematical Problems in Engineering from Hindawi
Bibliographic data for series maintained by Mohamed Abdelhakeem ().