Research map / Vision-language and multimodal / Cross-modal learning
Visual Agreement Regularized Training for Multi-Modal Machine Translation
Pengcheng Yang, Boxing Chen, Pei Zhang, Xu Sun
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply hard visual-agreement regularization | Seq2Seq MMT · Multi30K test-2017 EN→DE | BLEU: 27.1 -> 29.3 | Improved |
| apply soft visual-agreement regularization | Seq2Seq MMT · Multi30K test-2017 EN→DE | BLEU: 27.1 -> 29.2 | Improved |
| apply hard visual-agreement regularization | Transformer MMT · Multi30K test-2017 DE→EN | BLEU: 32.7 -> 33.4 | Improved |
| use soft rather than hard visual-agreement regularization | Transformer MMT · Multi30K test-2017 EN→DE | BLEU: 29.3 -> 29.5 | Improved |
| use soft rather than hard visual-agreement regularization | Transformer MMT · Multi30K test-2017 EN→FR | BLEU: 53.3 -> 52.8 | Hurt |
| add image features (ablation) | Seq2Seq MMT · Multi30K test-2017 EN→FR | BLEU: 51.3 -> 51.2 | Hurt |
| add multi-head co-attention (ablation) | Seq2Seq MMT · Multi30K test-2017 EN→DE | BLEU: 27.1 -> 28.6 | Improved |
| add multi-head co-attention (ablation) | Seq2Seq MMT · Multi30K test-2017 EN→FR | BLEU: 51.3 -> 51.7 | Improved |
| use adaptive rather than frozen regularization weights (ablation) | soft-agreement Seq2Seq MMT · Multi30K test-2017 EN→DE | BLEU: 28.8 -> 29.2 | Improved |
| use adaptive rather than frozen regularization weights (ablation) | soft-agreement Seq2Seq MMT · Multi30K test-2017 EN→FR | BLEU: 52.1 -> 52.4 | Improved |
| apply visual agreement regularization | bidirectional translation models · Multi30K EN→DE visual word pairs | VAD: 1.18 -> 0.74 | Improved |
Closest papers by meaning
- MMTIT-Bench: A Multilingual and Multi-Scenario Benchmark with Cognition-Perception-Reasoning Guided Text-Image Machine Translation (CVPR 2026 · similarity 0.87)
- Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization (ICLR 2024 · similarity 0.86)
- LRM-LLaVA: Overcoming the Modality Gap of Multilingual Large Language-Vision Model for Low-Resource Languages (AAAI 2025 · similarity 0.86)
- Hugging Visual Prompt and Segmentation Tokens: Consistency Learning for Fine-Grained Visual Understanding in MLLMs (CVPR 2026 · similarity 0.86)
- Multi-modal Instruction Tuned LLMs with Fine-grained Visual Perception (CVPR 2024 · similarity 0.86)
- Visual Perception by Large Language Model’s Weights (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
