Research map / Vision-language and multimodal / Multimodal LLMs
Bias-aware multi-image robustness
Analyzes language and position biases in LVLMs and corrects them through selective training or attention perturbation.
44 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2024–2026) · an approach in Multimodal LLMs, keyword cluster “hallucination · lvlms · mitigating”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
1 papers in 2023–24 and 43 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 0.5% to 2.0%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2024 · Unleashing Region Understanding in Intermediate Layers for MLLM-based Referring Expression Generation (NeurIPS 2024 · 1 citation)
- 2025 · Debiasing Multimodal Large Language Models via Noise-Aware Preference Optimization (CVPR 2025 · 9 citations)
- 2026 · Enhancing Medical Large Vision-Language Models via Alignment Distillation (AAAI 2026 · 2 citations)
Earliest papers here
- Unleashing Region Understanding in Intermediate Layers for MLLM-based Referring Expression Generation (NeurIPS 2024 · 1 citation)
- Debiasing Multimodal Large Language Models via Noise-Aware Preference Optimization (CVPR 2025 · 9 citations)
- Towards Understanding How Knowledge Evolves in Large Vision-Language Models (CVPR 2025 · 6 citations)
What has been tried
13 papers here report 147 measured changes (a component added, removed or swapped, compared with a baseline): helped 70%, no clear effect 4%, hurt 22%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| remove dynamic loss weighting | VLindBench · CB | 58.9 -> 50.0 | Hurt | Debiasing Multimodal Large Language Models via Noise-Aware Preference Optimization (CVPR 2025 · 9 citations) |
| Add trigger optimization | CLIP ViT-L, COCO · Sim-T | 0.658 -> 0.809 | Helped | Stealthy Backdoor Attack in Self-Supervised Learning Vision Encoders for Large Vision Language Models (CVPR 2025 · 5 citations) |
| Set interpolation weight sigma to 0 (pure inter-image causal attention) vs tuned sigma | BLINK · accuracy | 47.85 at sigma 0 vs 50.04 at sigma 0.50 | Hurt | Identifying and Mitigating Position Bias of Multi-image Vision-Language Models (CVPR 2025 · 3 citations) |
| Ablate object tokens and adjacent tokens (1 buffer) | COCO object identification, generative description, polling, and VQA · performance decrease (%) | 71.79; 51.28; 86.67 | Hurt | Towards Interpreting Visual Information Processing in Vision-Language Models (ICLR 2025 · 3 citations) |
| use self-retrospective context instead of no context | MSVD-QA out-domain · Acc. | 78.73 -> 80.36 | Helped | ISR-DPO: Aligning Large Multimodal Models for Videos by Iterative Self-Retrospective DPO (AAAI 2025 · 1 citation) |
| Add preference data selection rather than using all data | AMBER · HalRate | 25.1 -> 23.9 | Helped | RoVRM: A Robust Visual Reward Model Optimized via Auxiliary Textual Preference Data (AAAI 2025 · 1 citation) |
| add clinical relevance score weighting to Stage 1 | IU-Xray · Average score | 10.95 -> 15.66 | Helped | MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization (ICML 2025 · 1 citation) |
| use candidate layers [0, 7] rather than [24, 31] | RefCOCOg · METEOR | 127.0 -> 173.0 | Helped | Unleashing Region Understanding in Intermediate Layers for MLLM-based Referring Expression Generation (NeurIPS 2024 · 1 citation) |
| Omit the preliminary forward pass | MSCOCO CHAIR · CHAIR_S | 39.80 (± 2.3) -> 39.73 (± 2.15) | No clear effect | Enhancing Vision-Language Model Reliability with Uncertainty-Guided Dropout Decoding (NeurIPS 2025 · 0 citations) |
| Add Needle-Level optimization after Context-Level | MUIRBench Overall · accuracy | 44.8 -> 48.6 | Helped | Zooming from Context to Cue: Hierarchical Preference Optimization for Multi-Image MLLMs (NeurIPS 2025 · 0 citations) |
| Add visual preference optimization | ObjHal · response hallucination rate | 4.92 -> 9.19 | Helped | CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMs (ICLR 2025 · 0 citations) |
| Real-LOD re-aligned expression data added back to SigLIP-filtered pairs (config C vs B) | OmniLabel COCO / O365 / OI · AP-des | COCO 22.2->24.2, O365 28.5->32.4, OI 31.4->33.5 | Helped | Re-Aligning Language to Visual Objects with an Agentic Workflow (ICLR 2025 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 43 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- COCO · 8 of 43 papers
- POPE · 5 of 43 papers
- VQAv2 · 3 of 43 papers
- GQA · 3 of 43 papers
- AMBER · 3 of 43 papers
- MSCOCO · 3 of 43 papers
- SLAKE · 3 of 43 papers
- VQA-RAD · 3 of 43 papers
Baselines
- VCD · 7 of 43 papers
- DPO · 7 of 43 papers
- GPT-4V · 5 of 43 papers
- GPT-4o · 4 of 43 papers
- HA-DPO · 3 of 43 papers
- LLaVA-RLHF · 3 of 43 papers
- RLHF-V · 3 of 43 papers
- mDPO · 3 of 43 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 44 papers; from OpenAlex and the papers' affiliations).
- Chinese Academy of Sciences · 2 papers
- Tsinghua University · 2 papers
Use it
Download BibTeX of 20 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Decoding-time hallucination mitigation (91 papers)
- Preference tuning against hallucination (58 papers)
Elsewhere in the topic
- Chain-of-thought multimodal reasoning (332 papers)
- VLM capability analysis (264 papers)
- Multimodal LLM benchmarks (252 papers)
- Spatial reasoning in VLMs (207 papers)
- Instruction-tuned multimodal assistants (199 papers)
- Benchmarks and analysis of MLLMs (181 papers)
- Reinforcement learning for visual reasoning (163 papers)
- Interleaved image-text generation with MLLMs (153 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).