Contextual Sentence Similarity from News Articles

Nikhil Chaturvedi and Jigyasu Dubey

International Journal of Scientific Research in Computer Science, Engineering and Information Technology, 2024, vol. 10, issue 2, 24-37

Abstract: An important topic in the field of natural language processing is the measurement of sentence similarity. It's important to precisely gauge how similar two sentences are. Existing methods for determining sentence similarity challenge two problems Because sentence level semantics are not explicitly modelled at training, labelled datasets are typically small, making them insufficient for training supervised neural models; and there is a training-test gap for unsupervised language modelling (LM) based models to compute semantic scores between sentences. As a result, this task is performed at a lower level. In this paper, we suggest a novel paradigm to handle these two concerns by robotics method framework. The suggested robotics framework is built on the essential premise that a sentence's meaning is determined by its context and that sentence similarity may be determined by comparing the probabilities of forming two phrases given the same context. In an unsupervised way, the proposed approach can create high-quality, large-scale datasets with semantic similarity scores between two sentences, bridging the train-test gap to a great extent. Extensive testing shows that the proposed framework does better than existing baselines on a wide range of datasets.

Keywords: sentence similarity; BERT; deep learning; Cosine Similarity (search for similar items in EconPapers)
Date: 2024
Note: Article URL: https://ijsrcseit.com/home/article/view/CSEIT2390628
References: Add references at CitEc
Citations:

Downloads: (external link)
https://ijsrcseit.com/home/article/view/CSEIT2390628 Article URL (text/html)
https://ijsrcseit.com/home/article/download/CSEIT2390628/CSEIT2390628 Full text (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:jbh:ijsrcs:v10:y2024:i2:id:4

DOI: 10.32628/CSEIT2390628

Access Statistics for this article

More articles in International Journal of Scientific Research in Computer Science, Engineering and Information Technology from International Journal of Scientific Research in Computer Science, Engineering and Information Technology
Bibliographic data for series maintained by Pankaj Sharma (USA) ().