Research map / Vision-language and multimodal / Multimodal LLMs
Mini-Gemini: Mining the Potential of Multi-Modality Vision Language Models
Yanwei Li, Yuechen Zhang, Chengyao Wang, Zhisheng Zhong, Yixin Chen, Ruihang Chu, Shaoteng Liu, Jiaya Jia
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add patch info mining with ConvNeXt-L HR encoder (LR 224, HR 512) (ablation) | LLaVA-1.5 Vicuna-7B · TextVQA | accuracy: 54.1 -> 58.1 | Improved |
| add patch info mining (LR 336, HR 768) (ablation) | LLaVA-1.5 Vicuna-7B · TextVQA | accuracy: 58.2 -> 61.5 | Improved |
| add patch info mining (LR 336, HR 768) (ablation) | LLaVA-1.5 Vicuna-7B · MM-Vet | score: 31.1 -> 34.6 | Improved |
| raise HR resolution from 512 to 768 (LR 224) (ablation) | Mini-Gemini · MME | score: 1485.2 -> 1478.3 | Hurt |
| use ConvNeXt-B instead of ConvNeXt-L as HR encoder (ablation) | Mini-Gemini · MME | score: 1517.0 -> 1451.7 | Hurt |
| use ConvNeXt-XXL instead of ConvNeXt-L as HR encoder (ablation) | Mini-Gemini · TextVQA | accuracy: 61.5 -> 62.0 | No clear effect |
| use ConvNeXt-XXL instead of ConvNeXt-L as HR encoder (ablation) | Mini-Gemini · MM-Vet | score: 34.6 -> 33.8 | Hurt |
| add ShareGPT4V captions (ablation) | Mini-Gemini · TextVQA | accuracy: 61.5 -> 63.2 | Improved |
| remove TextCaps data (ablation) | Mini-Gemini · TextVQA | accuracy: 63.2 -> 59.0 | Hurt |
| add OCR-related data (ablation) | Mini-Gemini · TextVQA | accuracy: 58.7 -> 61.6 | Improved |
| add generation-related instructions (ablation) | Mini-Gemini · MM-Vet | score: 33.7 -> 37.0 | Improved |
| add ALLaVA GPT4V responses (ablation) | Mini-Gemini · MM-Vet | score: 37.0 -> 40.8 | Improved |
Later papers that compared with this method
3 later accepted papers report Mini-Gemini as a baseline in their results tables; 3 of them report a better number on most of the results they share with it.
- Deciphering Cross-Modal Alignment in Large Vision-Language Models via Modality Integration Rate (ICCV 2025 · better on 2 of 2 shared results)
- VisionUnite: A Vision-Language Foundation Model for Ophthalmology Enhanced With Clinical Knowledge (TPAMI 2025 · better on 2 of 2 shared results)
- Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders (ICLR 2025 · better on 1 of 1 shared result)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Are We on the Right Way for Evaluating Large Vision-Language Models? (NeurIPS 2024 · similarity 0.90)
- InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks (CVPR 2024 · similarity 0.90)
- Generalizing from SIMPLE to HARD Visual Reasoning: Can We Mitigate Modality Imbalance in VLMs? (ICML 2025 · similarity 0.90)
- Synthesize Diagnose and Optimize: Towards Fine-Grained Vision-Language Understanding (CVPR 2024 · similarity 0.90)
- Pink: Unveiling the Power of Referential Comprehension for Multi-modal LLMs (CVPR 2024 · similarity 0.89)
- HoneyBee: Data Recipes for Vision-Language Reasoners (CVPR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
