From bilingual to multilingual neural‐based machine translation by incremental training

Carlos Escolano, Marta R. Costa‐Jussà and José A. R. Fonollosa

Journal of the Association for Information Science & Technology, 2021, vol. 72, issue 2, 190-203

Abstract: A common intermediate language representation in neural machine translation can be used to extend bilingual systems by incremental training. We propose a new architecture based on introducing an interlingual loss as an additional training objective. By adding and forcing this interlingual loss, we can train multiple encoders and decoders for each language, sharing among them a common intermediate representation. Translation results on the low‐resource tasks (Turkish‐English and Kazakh‐English tasks) show a BLEU improvement of up to 2.8 points. However, results on a larger dataset (Russian‐English and Kazakh‐English) show BLEU losses of a similar amount. While our system provides improvements only for the low‐resource tasks in terms of translation quality, our system is capable of quickly deploying new language pairs without the need to retrain the rest of the system, which may be a game changer in some situations. Specifically, what is most relevant regarding our architecture is that it is capable of: reducing the number of production systems, with respect to the number of languages, from quadratic to linear; incrementally adding a new language to the system without retraining the languages already there; and allowing for translations from the new language to all the others present in the system.

Date: 2021
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1002/asi.24395

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:jinfst:v:72:y:2021:i:2:p:190-203

Ordering information: This journal article can be ordered from
http://www.blackwell ... bs.asp?ref=2330-1635

Access Statistics for this article

More articles in Journal of the Association for Information Science & Technology from Association for Information Science & Technology
Bibliographic data for series maintained by Wiley Content Delivery ().