An Oblivious Approach to Machine Translation Quality Estimation

Itamar Elmakias and Dan Vilenchik
Additional contact information
Itamar Elmakias: Department of Industrial Engineering & Management, Ben-Gurion University of the Negev, Beer-Sheva P.O. Box 653, Israel
Dan Vilenchik: School of Electrical and Computer Engineering, Ben-Gurion University of the Negev, Beer-Sheva P.O. Box 653, Israel

Mathematics, 2021, vol. 9, issue 17, 1-11

Abstract: Machine translation (MT) is being used by millions of people daily, and therefore evaluating the quality of such systems is an important task. While human expert evaluation of MT output remains the most accurate method, it is not scalable by any means. Automatic procedures that perform the task of Machine Translation Quality Estimation (MT-QE) are typically trained on a large corpus of source–target sentence pairs, which are labeled with human judgment scores. Furthermore, the test set is typically drawn from the same distribution as the train. However, recently, interest in low-resource and unsupervised MT-QE has gained momentum. In this paper, we define and study a further restriction of the unsupervised MT-QE setting that we call oblivious MT-QE. Besides having no access no human judgment scores, the algorithm has no access to the test text’s distribution. We propose an oblivious MT-QE system based on a new notion of sentence cohesiveness that we introduce. We tested our system on standard competition datasets for various language pairs. In all cases, the performance of our system was comparable to the performance of the non-oblivious baseline system provided by the competition organizers. Our results suggest that reasonable MT-QE can be carried out even in the restrictive oblivious setting.

Keywords: machine translation; unsupervised learning; quality estimation (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2021
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/9/17/2090/pdf (application/pdf)
https://www.mdpi.com/2227-7390/9/17/2090/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:9:y:2021:i:17:p:2090-:d:624707

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().