Research map / Vision-language and multimodal / Multimodal LLMs
Chimera: Improving Generalist Model with Domain-Specific Experts
Tianshuo Peng, Mingsheng Li, Jiakang Yuan, Hongbin Zhou, Renqiu Xia, Renrui Zhang, Lei Bai, Song Mao, Bin Wang, Aojun Zhou, Botian Shi, Tao Chen and 2 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Generalist-Specialist Collaboration Masking at ratio 0.3 (ablation) | Chimera-4B · MathVista testmini | accuracy: 59.4 -> 61.3 | Improved |
| raise GSCM mask ratio from 0.3 to 0.5 (ablation) | Chimera-4B · MathVista testmini | accuracy: 61.3 -> 60.4 | Hurt |
| raise GSCM mask ratio from 0.3 to 1.0 (ablation) | Chimera-4B · MathVista testmini | accuracy: 61.3 -> 56.2 | Hurt |
| add expert encoders without masking instead of naive finetune (ablation) | InternVL2-4B · MathVista testmini | accuracy: 58.5 -> 59.4 | Improved |
| naively finetune baseline on same data (ablation) | InternVL2-4B · MathVista testmini | accuracy: 57.0 -> 58.5 | Improved |
| add table and math experts to chart-only expert (ablation) | InternVL2-4B · MathVista testmini | accuracy: 59.4 -> 61.3 | Improved |
| add Chimera experts to InternVL2-8B | InternVL2-8B · MathVista testmini | accuracy: 61.6 -> 64.9 | Improved |
| add Chimera experts to InternVL2-2B | InternVL2-2B · MathVista testmini | accuracy: 48.3 -> 53.1 | Improved |
| add Chimera experts to InternVL2-8B | InternVL2-8B · MathVerse | accuracy: 31.3 -> 32.4 | Improved |
| apply direct preference optimization on 60K pairs | Chimera-8B · MathVista testmini | accuracy: 64.9 -> 68.3 | Improved |
| add experts to InternVL2-8B in the table domain | InternVL2-8B · MathVista table domain | accuracy: 67.1 -> 62.9 | Hurt |
| add experts to InternVL2-8B in the math domain | InternVL2-8B · MathVista math domain | accuracy: 66.5 -> 71.9 | Improved |
Closest papers by meaning
- L-Man: A Large Multi-modal Model Unifying Human-centric Tasks (AAAI 2025 · similarity 0.87)
- Cross-modal Representation Flattening for Multi-modal Domain Generalization (NeurIPS 2024 · similarity 0.87)
- MMA: Multi-Modal Adapter for Vision-Language Models (CVPR 2024 · similarity 0.87)
- OptMerge: Unifying Multimodal LLM Capabilities and Modalities via Model Merging (ICLR 2026 · similarity 0.87)
- LVLM-Aided Alignment of Task-Specific Vision Models (CVPR 2026 · similarity 0.86)
- On the Out-Of-Distribution Generalization of Large Multimodal Models (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
