Research map / Vision-language and multimodal / Multimodal LLMs
VIVAS: Vitalizing Visual Perception in VLM Pre-training via Vision-language Unified Autoregressive Supervision
Zhehan Kan, Yubo Zhu, Xinghua Jiang, Zhixiang Wei, Shifeng Liu, Wei Tong, Sheng Zhong, Qingmin Liao, Wenming Yang, Xin Li, Yinsong Liu, Deqiang Jiang and 1 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add dense-structural-semantic visual supervision in pre-training (ablation) | VLM (0.5T) · VisuLogic (0.5T, 5-shot) | accuracy: 22.2 -> 24.4 | Improved |
| add dense-structural-semantic visual supervision in pre-training (ablation) | VLM (0.5T) · RealWorldQA (0.5T, 5-shot) | accuracy: 43.0 -> 46.5 | Improved |
| add text-decoupled (VQ-VAE) visual vocabulary supervision (ablation) | VLM (0.5T) · MMBench-EN (0.5T, 5-shot) | accuracy: 53.0 -> 51.2 | Hurt |
| add vision-language semantic (SigLIP2) vocabulary supervision (ablation) | VLM (0.5T) · MMBench-EN (0.5T, 5-shot) | accuracy: 53.0 -> 53.6 | No clear effect |
| add visual supervision in post-training only (ablation) | VIVAS 4B · VisuLogic | accuracy: 23.1 -> 23.3 | No clear effect |
| add visual supervision in pre-training only (ablation) | VIVAS 4B · VisuLogic | accuracy: 23.1 -> 25.7 | Improved |
| also add visual supervision in post-training on top of pre-training (ablation) | VIVAS 4B · VisuLogic | accuracy: 25.7 -> 23.6 | Hurt |
| add L1 reconstruction loss to codebook training (ablation) | VIVAS tokenizer · VisuLogic (0.5T, 5-shot) | accuracy: 24.4 -> 23.7 | Hurt |
| increase codebook size from 50k to 150k (ablation) | VIVAS tokenizer · VisuLogic (0.5T, 5-shot) | accuracy: 23.6 -> 24.4 | Improved |
| increase codebook size from 150k to 200k (ablation) | VIVAS tokenizer · VisuLogic (0.5T, 5-shot) | accuracy: 24.4 -> 24.2 | Hurt |
| increase codebook dim from 64 to 768 (ablation) | VIVAS tokenizer · VisuLogic (0.5T, 5-shot) | accuracy: 22.7 -> 24.4 | Improved |
| replace visual supervision with combined SigLIP2+DINOv3 input encoder (ablation) | VLM (0.5T) · RefCOCO val (0.5T, 5-shot) | accuracy: 62.1 -> 60.9 | Hurt |
Closest papers by meaning
- UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm (CVPR 2026 · similarity 0.92)
- Accelerating Pre-training of Multimodal LLMs via Chain-of-Sight (NeurIPS 2024 · similarity 0.90)
- LION: Empowering Multimodal Large Language Model with Dual-Level Visual Knowledge (CVPR 2024 · similarity 0.89)
- CoVFT: Context-aware Visual Fine-tuning for Multimodal Large Language Models (CVPR 2026 · similarity 0.89)
- VILA: On Pre-training for Visual Language Models (CVPR 2024 · similarity 0.89)
- Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
