Research map / Vision-language and multimodal / Multimodal LLMs
UniTranslator: A Unified Multi-modal framework for End-to-end In-Image Machine Translation
Jiahao Lyu, Pei Fu, Zhenhang Li, Shaojie Zhang, Jiahui Yang, Yu ZHOU, Can Ma, Zhenbo Luo, Jian Luan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add UGAM and SMD to the baseline (ablation) | UniTranslator · Translatotron-V De->En | Structure-BLEU: 9.00 -> 24.86 | Improved |
| add UGAM alone (ablation) | UniTranslator · Translatotron-V De->En | Structure-BLEU: 9.00 -> 14.28 | Improved |
| add UGAM alone (ablation) | UniTranslator · Translatotron-V De->En | SSIM: 0.8081 -> 0.8036 | Hurt |
| add SMD alone (ablation) | UniTranslator · Translatotron-V De->En | Structure-BLEU: 9.00 -> 16.02 | Improved |
| add SMD alone (ablation) | UniTranslator · Translatotron-V De->En | SSIM: 0.8081 -> 0.8138 | Improved |
| add Stage 2 joint fine-tuning to Stage 1 only (ablation) | UniTranslator · Translatotron-V De->En | BLEU: 12.53 -> 25.03 | Improved |
| add Stage 1 warm-up before Stage 2 (ablation) | UniTranslator · Translatotron-V De->En | BLEU: 15.08 -> 25.03 | Improved |
| add Stage 1 warm-up before Stage 2 (ablation) | UniTranslator · Translatotron-V De->En | SSIM: 0.8148 -> 0.8184 | Improved |
| add understanding loss to generation training (ablation) | UniTranslator · Translatotron-V | Structure-BLEU: 16.57 -> 25.03 | Improved |
| add generation loss to understanding training (ablation) | UniTranslator · Translatotron-V | BLEU: 25.70 -> 34.71 | Improved |
| add generation loss to understanding training (ablation) | UniTranslator · Translatotron-V | COMET (%): 80.84 -> 84.61 | Improved |
| add UGAM to baseline on scene text editing (ablation) | UniTranslator · TextEditBench | SSIM: 0.8709 -> 0.8928 | Improved |
Closest papers by meaning
- MMTIT-Bench: A Multilingual and Multi-Scenario Benchmark with Cognition-Perception-Reasoning Guided Text-Image Machine Translation (CVPR 2026 · similarity 0.88)
- MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer (ICLR 2026 · similarity 0.87)
- Vision as a Dialect: Unifying Visual Understanding and Generation via Text-Aligned Representations (NeurIPS 2025 · similarity 0.87)
- Towards Unified Multimodal Interleaved Generation via Group Relative Policy Optimization (NeurIPS 2025 · similarity 0.86)
- OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation (CVPR 2025 · similarity 0.86)
- The Narrow Gate: Localized Image-Text Communication in Native Multimodal Models (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
