Research map / Vision-language and multimodal / Multimodal LLMs
Improving Large Vision and Language Models by Learning from a Panel of Peers
Jefferson Hernandez, Jing Shi, Simon Jenni, Vicente Ordonez, Kushal Kafle
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| run three iterations of self-improvement instead of one | PoP-Vicuna · 15 vision-language benchmarks (average) | average score: 53.1 -> 56.7 | Improved |
| run three iterations instead of one | PoP LLaVA-1.5 · ScienceQA | accuracy: 71.2 -> 86.4 | Improved |
| use full panel instead of single-try panel (st-PoP) | PoP LLaVA-1.5 iter3 · MMBench | accuracy: 67.4 -> 72.5 | Improved |
| use full panel instead of single-try panel (st-PoP) | PoP LLaVA-1.5 iter3 · POPE | accuracy: 86.3 -> 87.0 | No clear effect |
| PoP-iter1 vs CSR (prior preference method) | PoP LLaVA-1.5 · MMBench | accuracy: 65.4 -> 68.7 | Improved |
| PoP-iter1 vs SeVa | PoP LLaVA-1.5 · MM-Vet | accuracy: 37.2 -> 34.1 | Hurt |
| train on peer-generated preference data instead of ground-truth SFT on 900K samples | PoP-Vicuna · 15 vision-language benchmarks (average) | average score: 54.0 -> 57.0 | Improved |
| add self-evaluation by each panel member (ablation) | PoP · 15 vision-language benchmarks (average) | average score | No clear effect |
| use absolute scoring instead of relative scoring (ablation) | PoP · 15 vision-language benchmarks (average) | average score | Improved |
| use uncertainty-weighted aggregation instead of mean (ablation) | PoP · 15 vision-language benchmarks (average) | average score | No clear effect |
| use panel-wide ensemble instead of average single model for zero-shot selection | PoP · Best-of-15 selection at four panel scales | average score | Improved |
| add more OCR data (0% to 100%) to OCR-Dumb panel member (ablation) | PoP · five capability categories | average score | Improved |
Closest papers by meaning
- Fine-Grained Verifiers: Preference Modeling as Next-token Prediction in Vision-Language Alignment (ICLR 2025 · similarity 0.89)
- Calibrated Self-Rewarding Vision Language Models (NeurIPS 2024 · similarity 0.88)
- LVLM-Aided Alignment of Task-Specific Vision Models (CVPR 2026 · similarity 0.88)
- DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedback (CVPR 2024 · similarity 0.88)
- Enhancing Large Vision Language Models with Self-Training on Image Comprehension (NeurIPS 2024 · similarity 0.87)
- Self-alignment of Large Video Language Models with Refined Regularized Preference Optimization (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
