Language Modelling Approaches to Adaptive Machine Translation
(2401.14559)Abstract
Consistency is a key requirement of high-quality translation. It is especially important to adhere to pre-approved terminology and adapt to corrected translations in domain-specific projects. Machine translation (MT) has achieved significant progress in the area of domain adaptation. However, in-domain data scarcity is common in translation settings, due to the lack of specialised datasets and terminology, or inconsistency and inaccuracy of available in-domain translations. In such scenarios where there is insufficient in-domain data to fine-tune MT models, producing translations that are consistent with the relevant context is challenging. While real-time adaptation can make use of smaller amounts of in-domain data to improve the translation on the fly, it remains challenging due to supported context limitations and efficiency constraints. LLMs have recently shown interesting capabilities of in-context learning, where they learn to replicate certain input-output text generation patterns, without further fine-tuning. Such capabilities have opened new horizons for domain-specific data augmentation and real-time adaptive MT. This work attempts to address two main relevant questions: 1) in scenarios involving human interaction and continuous feedback, can we employ language models to improve the quality of adaptive MT at inference time? and 2) in the absence of sufficient in-domain data, can we use pre-trained large-scale language models to improve the process of MT domain adaptation?
Overview
-
The paper discusses the WMT 2023 Terminology Shared Task, which aims to improve machine translation of technical terminology.
-
Researchers designed systems for DE-EN, EN-CS, and ZH-EN translations using LLMs for generating synthetic bilingual data and fine-tuning pre-existing MT models.
-
Terminology-constrained automatic post-editing was used to incorporate missing terms in the translation without changing the rest of the text.
-
The approach significantly increased the usage of specified terms in translations and maintained overall translation quality.
-
Future work will expand methods to more languages and domains, and explore LLMs in post-editing tasks to optimize efficiency.
Enhancing Machine Translation with LLMs: Domain Terminology Integration
Overview of the WMT 2023 Terminology Shared Task
In the pursuit of advancing machine translation (MT), the WMT 2023 Terminology Shared Task stimulates researchers to design systems capable of translating technical terminology with high accuracy. This year's challenge emphasizes the translation's adherence to specified technical terms, which is crucial in domain-specific communications.
Approach to Terminology Integration
Our systems for the task, centered around German-to-English (DE-EN), English-to-Czech (EN-CS), and Chinese-to-English (ZH-EN) pairs, harnessed LLMs for two vital operations. Firstly, we generated synthetic bilingual data informed by the required terminology using ChatGPT. Subsequently, we fine-tuned a pre-existing generic OPUS MT model with a composite of the synthetic data alongside a random selection of the OPUS generic dataset. Post the fine-tuning, we used the refined MT model for producing translations of the datasets furnished by the task organizers.
Terminology-Constrained Automatic Post-Editing
For translations omitting any requisite terms, we deployed a terminology-constrained automatic post-editing step employing ChatGPT. This step revised the MT output to include overlooked terminology, endeavoring to fulfill the task's terminology constraints without altering the untranslated portions of text.
Results and Findings
Our process demonstrated its efficacy by significantly increasing the prevalence of desired terms in the final translations. Notably, the translations of the blind dataset witnessed an increase in the usage of the specified terms from 36.67% with the original model to an impressive 72.88% following the LLM-based editing—effectively doubling the adherence rate across the three language pairs.
Our evaluation was two-pronged: a term-level evaluation showed the increased fidelity to the required terms, while a sentence-level evaluation assessed whether the term integration affected the overall translation integrity. The automatic evaluations validate that our system improves translations in terms of both terminology adherence and overall quality.
In scenarios where the translation quality by an LLM is considerably weaker than that of a traditional MT model, starting with the stronger MT baseline and seeking improvements proved beneficial. Crucially, the success of this process is dependent on the level of language support provided by the respective LLM. Furthermore, real-time adaptive MT is not a substitute for domain-specific fine-tuning, thus using fine-tuning where possible ensures efficiency—reducing the need for post-editing at inference time.
Future Directions
Future work will look to extend these methods to additional languages and domains, especially low-resource pairs. The techniques discussed may also omit the fine-tuning phase to understand if merely employing LLMs for post-editing tasks is sufficient for quality translation, optimizing the translation process for efficiency and reducing latency at inference.
Overall, the systematic approach of combining domain-specific fine-tuning and LLM-powered terminology editing provides robust solutions for enhancing domain-specific machine translation workflows, thereby contributing to the development of more accurate and reliable translation systems.
Create an account to read this summary for free: