Research map / Vision-language and multimodal / Multimodal LLMs
EVEv2: Improved Baselines for Encoder-Free Vision-Language Models
Haiwen Diao, Xiaotong Li, Yufeng Cui, Yueze Wang, Haoge Deng, Ting Pan, Wenxuan Wang, Huchuan Lu, Xinlong Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fully decouple attention, LayerNorm and FFN per modality (vs decoupling only LayerNorm) (ablation) | EVEv2.0 (Qwen2.5-7B) · Avg. over benchmarks, 8M data | Avg. accuracy: 48.8% -> 51.6% | Improved |
| use Divide-and-Conquer (full decoupling) instead of MoE (FFN-only) as data grows (ablation) | EVEv2.0 vs EVEv1.5 · Avg. over benchmarks, 8M to 24M data | Avg. accuracy gap: 0.8% -> 1.4% | Improved |
| full modality decoupling vs MoE vs dense on Qwen2.5-1.5B (ablation) | EVE variants · Six VL benchmarks, EVE-recap-4M | Avg. score: 35.1% -> 37.3% -> 38.5% | Improved |
| full decoupling vs MoE vs ReP (SFT loss) (ablation) | EVEv1.2, EVEv1.5, EVEv2 · SFT stage | SFT loss: 1.0306 -> 0.9513 -> 0.9327 | Improved |
| re-parameterization (update LLM weights) instead of frozen LLM with vision layers (ablation) | EVEv1.2 · SQA-IMG | accuracy | Hurt |
| add modality-specific weights (EVEv2.0) vs EVEv1.5 MoE (ablation) | EVEv1.5 · All multimodal benchmarks, Stage 2 | accuracy | Improved |
| recaption data with DenseFusion++ (7B) vs LLaVA-1.5 (13B)/Emu2 (17B) captions or raw web captions (ablation) | EVEv1.0 · Pretraining data comparison | benchmark accuracy | Improved |
| mix LAION, OpenImages and SAM data into pretraining (ablation) | EVEv1.0 · Pretraining data mixture | benchmark accuracy | Improved |
| use AnyRatio_HD (fixed 1600x1600 area) image processing (ablation) | EVEv1.0 (Vicuna-7B) · Image input formats | benchmark accuracy | Improved |
| use AnyResolution input (ablation) | EVEv1.0 (Vicuna-7B) · Image input formats | benchmark accuracy | Mixed |
| add modality-specific weights (EVEv2.0 vs EVEv1.0 dense) (ablation) | EVEv1.0 · TTFT | time to first token (s): 3.77 -> 4.25 | Hurt |
| unfreeze LLM when increasing captioning data (ablation) | EVE variants · SQA-IMG | accuracy | Hurt |
Closest papers by meaning
- Unveiling Encoder-Free Vision-Language Models (NeurIPS 2024 · similarity 0.96)
- Eve: Efficient Multimodal Vision Language Models with Elastic Visual Experts (AAAI 2025 · similarity 0.92)
- Beyond Encoder Accumulation: Measuring Encoder Roles in Multi-Encoder VLMs (NeurIPS 2026 · similarity 0.91)
- EVE: Efficient Vision-Language Pre-training with Masked Prediction and Modality-Aware MoE (AAAI 2024 · similarity 0.90)
- From Pixels to Words -- Towards Native Vision-Language Primitives at Scale (ICLR 2026 · similarity 0.90)
- Rethinking Model Selection in VLM Through the Lens of Gromov-Wasserstein Distance (CVPR 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
