Research map / Large language models / Language, safety and interpretability
DelTA: An Online Document-Level Translation Agent Based on Multi-Level Memory
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DelTA, an online sentence-by-sentence document translation agent that uses a four-level memory (proper noun records, bilingual summary, long-term and short-term memory) maintained by LLM components to improve terminology consistency and quality.
Problem
Document-level machine translation with LLMs suffers from proper noun translation inconsistency, sentence omissions when translating multi-sentence windows, and inaccurate translations for long documents.
Approach
Translate each sentence separately while retrieving and updating a multi-level memory (proper noun dictionary, bilingual source/target summaries, long-term retrieved sentence pool, short-term recent sentence pairs) via auxiliary LLM components, which are injected into the translator prompt.
Key results
- On IWSLT2017 averaged over four backbones, DelTA reaches LTCR-1 of 81.63 (En->Xx) and 85.09 (Xx->En) versus 79.02 and 83.44 for the best Context baseline, with COMET gains up to 3.16 points.
- On Guofeng (Zh->En), with Qwen2-72B-Instruct DelTA raises LTCR-1 from 58.21 (Context) to 86.93, and with Qwen2-7B from 37.00 (Sentence) to 85.50.
- On QMSum, DelTA achieves ROUGE-L 23.60 vs 21.50 for ReadAgent.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Context with DelTA's multi-level memory translation agent | GPT-3.5-Turbo, four-model average · IWSLT2017 En ⇒ Xx | sCOMET: 82.95 -> 84.36 | Helped |
| replace Context with DelTA's multi-level memory translation agent | GPT-3.5-Turbo, four-model average · IWSLT2017 En ⇒ Xx | LTCR-1: 79.02 -> 81.63 | Helped |
| replace Context with DelTA's multi-level memory translation agent | GPT-3.5-Turbo, four-model average · IWSLT2017 Xx ⇒ En | dCOMET: 7.03 -> 7.11 | Helped |
| replace Context with DelTA's multi-level memory translation agent | GPT-3.5-Turbo, four-model average · IWSLT2017 Xx ⇒ En | LTCR-1_f: 93.59 -> 95.48 | Helped |
| replace Context with DelTA | GPT-3.5-Turbo · Guofeng V1 TEST_2 Zh ⇒ En | sCOMET: 78.57 -> 78.45 | Hurt |
| replace Context with DelTA | GPT-3.5-Turbo · Guofeng V1 TEST_2 Zh ⇒ En | LTCR-1: 70.10 -> 85.57 | Helped |
| add Short-Term Memory (ablation) | sentence-level GPT-3.5-Turbo-0125 · IWSLT2017 En ⇒ Zh | sCOMET: 83.78 -> 84.50 | Helped |
| add Short-Term Memory (ablation) | sentence-level GPT-3.5-Turbo-0125 · IWSLT2017 En ⇒ Zh | LTCR-1: 80.27 -> 77.89 | Hurt |
| add Long-Term Memory to Short-Term Memory (ablation) | GPT-3.5-Turbo-0125 · IWSLT2017 En ⇒ Zh | sCOMET: 84.50 -> 84.54 | Helped |
| add Long-Term Memory to Short-Term Memory (ablation) | GPT-3.5-Turbo-0125 · IWSLT2017 En ⇒ Zh | LTCR-1: 77.89 -> 79.23 | Helped |
| add Source Summary (ablation) | Short-Term and Long-Term Memory GPT-3.5-Turbo-0125 · IWSLT2017 En ⇒ Zh | LTCR-1: 79.23 -> 76.09 | Hurt |
| add Target Summary (ablation) | Short-Term and Long-Term Memory GPT-3.5-Turbo-0125 · IWSLT2017 En ⇒ Zh | sCOMET: 84.54 -> 84.70 | Helped |
Surprising findings
- Short-term memory and long-term memory individually improve quality but lower proper noun consistency, while combining them recovers consistency.
- Adding proper noun records slightly lowered sCOMET and dCOMET in the ablation despite raising consistency.
Limitations
- High inference cost and latency from frequent LLM invocations for auxiliary components.
- Proper noun extraction and alignment rely on LLMs and spaCy/awesome-align, which introduce errors and language coverage limits.
- Gains in consistency are smaller for language pairs sharing an alphabet such as En->De.
Experimental setup
Datasets: IWSLT2017 tst2017 Guofeng Webnovel V1 TEST_2 Europarl v9 (low-resource Lt-En sample) QMSum CTXPRO mini.gender.opensubtitles
Benchmarks: IWSLT2017 Guofeng Webnovel QMSum CTXPRO
Metrics: sCOMET dCOMET LTCR-1 LTCR-1_f APT ROUGE-L
Baselines: Sentence-level prompting Context (3 previous sentence pairs) Doc2Doc NLLB-3.3B GoogleTrans ReadAgent
Builds on: ReadAgent Doc2Doc document translation Context-aware prompting for DocMT
Closest papers by meaning
- A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models (ICLR 2024 · similarity 0.90)
- A Benchmark for Learning to Translate a New Language from One Grammar Book (ICLR 2024 · similarity 0.86)
- Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation (ICML 2024 · similarity 0.86)
- BenchmarkCards: Standardized Documentation for Large Language Model Benchmarks (NeurIPS 2025 · similarity 0.86)
- Calibrating Translation Decoding with Quality Estimation on LLMs (NeurIPS 2025 · similarity 0.86)
- Teaching Large Language Models to Translate with Comparison (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.