Research map / Vision-language and multimodal / CLIP and zero-shot
uCLIP: Parameter-Efficient Multilingual Extension of Vision-Language Models with Unpaired Data
Dahyun Chung, Donghyun Shin, Yujin Sung, Seunggi Moon, Jinwoo Jeon, Byung-Jun Lee
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add embedding perturbation (Gaussian noise) (ablation) | uCLIP · MSCOCO (translated) | I2T R@10: 32.6 -> 53.2 | Improved |
| add embedding perturbation (Gaussian noise) (ablation) | uCLIP · MSCOCO (translated) | T2I R@10: 48.6 -> 53.3 | Improved |
| add pseudo-pair alignment loss Lpseudo (ablation) | uCLIP · MSCOCO (translated) | I2T R@10: 47.7 -> 53.2 | Improved |
| add pseudo-pair alignment loss Lpseudo (ablation) | uCLIP · MSCOCO (translated) | T2I R@10: 52.4 -> 53.3 | No clear effect |
| add text alignment loss Ltext (ablation) | uCLIP · MSCOCO (translated) | I2T R@10: 51.2 -> 53.2 | Improved |
| add intra-alignment loss Lintra (ablation) | uCLIP · MSCOCO (translated) | I2T R@10: 52.5 -> 53.2 | No clear effect |
| use MPNet-base instead of multilingual E5-base text encoder (ablation) | uCLIP · MSCOCO (translated) | I2T R@10: 32.2 -> 53.2 | Improved |
| use MPNet-base instead of MiniLM-L12 text encoder (ablation) | uCLIP · MSCOCO (translated) | I2T R@10: 48.9 -> 53.2 | Improved |
| apply uCLIP projector to SigLIP2 image encoder (ablation) | uCLIP · MSCOCO (translated) | I2T R@10: 44.7 -> 54.6 | Improved |
| uCLIP vs M-CLIP on text-to-image retrieval | uCLIP · XM3600 (5 langs) | T2I R@10: 77.5 -> 70.4 | Hurt |
| uCLIP vs M-CLIP on zero-shot classification | uCLIP · STL-10 (5 langs) | F1: 94.3 -> 91.1 | Hurt |
| skip translation and use uCLIP instead of translate-then-CLIP | uCLIP · inference latency | ms/sample: 480.13 -> 23.56 | Improved |
Closest papers by meaning
- Multilingual Diversity Improves Vision-Language Representations (NeurIPS 2024 · similarity 0.92)
- Harnessing Frozen Unimodal Encoders for Flexible Multimodal Alignment (CVPR 2025 · similarity 0.91)
- Embracing Language Inclusivity and Diversity in CLIP through Continual Language Learning (AAAI 2024 · similarity 0.91)
- LRM-LLaVA: Overcoming the Modality Gap of Multilingual Large Language-Vision Model for Low-Resource Languages (AAAI 2025 · similarity 0.91)
- DiM$^3$: Bridging Multilingual and Multimodal Models via Direction- and Magnitude-Aware Merging (NeurIPS 2026 · similarity 0.90)
- Assessing and Learning Alignment of Unimodal Vision and Language Models (CVPR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
