Research map / Vision-language and multimodal / Multimodal LLMs
Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs
Sanghwan Kim, Rui Xiao, Stephan Alaniz, Yongqin Xian, Zeynep Akata
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add UG-Search (lowest-entropy crop selection) to the MLLM | InternVL2.5-8B · V* Bench | accuracy: 71.7 -> 91.1 | Improved |
| add UG-Search to standard QA with regular-size images | Qwen2.5-VL-7B · TextVQA | accuracy: 76.1 -> 79.7 | Improved |
| add UG-Sample (entropy-based top-8 frame selection) instead of uniform sampling | Qwen2.5-VL-7B · Video-MME (overall) | accuracy: 53.7 -> 59.8 | Improved |
| add UG-Sample on short-video benchmarks | Qwen2.5-VL-7B · EgoSchema | accuracy: 53.7 -> 56.6 | Improved |
| add UG-Ground (BRC score with Kadane search) to the MLLM | LLaVA-OV-7B · Charades-STA | mIoU: 10.4 -> 46.8 | Improved |
| replace AKS CLIP scoring with entropy score (UG-Sample+AKS hybrid) for coverage | UG-Sample · MLVU | accuracy: 62.0 -> 65.0 | Improved |
| use a larger scoring MLLM (26B instead of 1B) with an 8B answerer (ablation) | UG-Search · V* Bench | accuracy: 75.4 -> 92.0 | Improved |
| use a larger scoring MLLM (26B instead of 1B) with an 8B answerer (ablation) | UG-Sample · Video-MME | accuracy: 58.5 -> 61.8 | Improved |
| add YOLOv8 pre-filter to UG-Search (ablation) | UG-Search on LLaVA-OV-7B · V* Bench | accuracy: 81.2 -> 74.9 | Hurt |
| add SAM2 pre-filter to UG-Search (ablation) | UG-Search on LLaVA-OV-7B · V* Bench | accuracy: 81.2 -> 78.2 | Hurt |
| add SigLIP pre-filter to UG-Sample (top-32 frames) (ablation) | UG-Sample on LLaVA-OV-7B · Video-MME | accuracy: 58.6 -> 57.3 | Mixed |
| add DINOv2 pre-filter to UG-Sample (top-32 frames) (ablation) | UG-Sample on LLaVA-OV-7B · Video-MME | accuracy: 58.6 -> 56.4 | Mixed |
Closest papers by meaning
- Be Confident: Uncovering Overfitting in MLLM Multi-Task Tuning (ICML 2025 · similarity 0.91)
- BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception (ICML 2026 · similarity 0.91)
- FUSE: Quantifying Uncertainty in Vision-Language Models by Bayesian Fusing Epistemic and Aleatoric Uncertainty (ICML 2026 · similarity 0.90)
- CoVFT: Context-aware Visual Fine-tuning for Multimodal Large Language Models (CVPR 2026 · similarity 0.90)
- Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment (CVPR 2025 · similarity 0.90)
- Exploiting the Asymmetric Uncertainty Structure of Pre-trained VLMs on the Unit Hypersphere (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
