SIT at MixMT 2022: Fluent Translation Built on Giant Pre-trained Models

Abstract: This paper describes the Stevens Institute of Technology's submission for the WMT 2022 Shared Task: Code-mixed Machine Translation (MixMT). The task consisted of two subtasks, subtask $1$ Hindi/English to Hinglish and subtask $2$ Hinglish to English translation. Our findings lie in the improvements made through the use of large pre-trained multilingual NMT models and in-domain datasets, as well as back-translation and ensemble techniques. The translation output is automatically evaluated against the reference translations using ROUGE-L and WER. Our system achieves the $1<sup>{st}$ position on subtask $2$ according to ROUGE-L, WER, and human evaluation, $1<sup>{st}$ position on subtask $1$ according to WER and human evaluation, and $3<sup>{rd}$ position on subtask $1$ with respect to ROUGE-L metric.