Research map / Vision-language and multimodal / Visual question answering
Human Uncertainty-Aware Data Selection and Automatic Labeling in Visual Question Answering
Jian Lan, Zhicheng Liu, Udo Schlegel, Raoyuan Zhao, Yihong Liu, Hinrich Schuetze, Michael A. Hedderich, Thomas Seidl
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add HU-aware Selectors (vs random samples) (ablation) | HaDola · VQAv2, Qwen2.5VL-7B | HU-acc: 72.23 -> 76.75 | Improved |
| add Error Trigger (ablation) | HaDola · VQAv2, Qwen2.5VL-7B | HU-acc: 71.56 -> 76.75 | Improved |
| use self-annotation instead of human labels (ablation) | HaDola · VQAv2, Qwen2.5VL-7B | HU-acc: 73.91 -> 76.75 | Improved |
| use HaDola loss instead of cross-entropy (ablation) | HaDola · VQAv2, Qwen2.5VL-7B | HU-acc: 72.37 -> 76.75 | Improved |
| train on high-HU samples vs low-HU samples | SFT · VQAv2 | HU-acc | Hurt |
| add high-HU data to low+medium HU training | SFT · VQAv2 | HU-acc | Hurt |
| combine low- and medium-HU samples | SFT · VQAv2 | HU-acc | Improved |
| use HU-acc instead of VQA-acc to evaluate subsets | Evaluation metric · VQAv2 | metric separation | Improved |
| HaDola with 5% labels vs SFT with 100% annotations | HaDola · VQAv2/VizWiz, zero-shot VLMs | HU-acc | Improved |
| HaDola on BEiT3 | HaDola · VQAv2/VizWiz, BEiT3 | HU-acc | No clear effect |
| increase labeled fraction beyond 15% | HaDola · VQAv2, VizWiz | HU-acc | No clear effect |
| change (beta, lambda) weights (ablation) | HaDola · VQAv2, Qwen2.5VL-7B | HU-acc: 76.75 -> 68.43 (worst) | Mixed |
Closest papers by meaning
- Mind the Uncertainty in Human Disagreement: Evaluating Discrepancies Between Model Predictions and Human Responses in VQA (AAAI 2025 · similarity 0.94)
- Consistency and Uncertainty: Identifying Unreliable Responses From Black-Box Vision-Language Models for Selective Visual Question Answering (CVPR 2024 · similarity 0.91)
- FRAMES-VQA: Benchmarking Fine-Tuning Robustness across Multi-Modal Shifts in Visual Question Answering (CVPR 2025 · similarity 0.90)
- Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models (ICLR 2024 · similarity 0.90)
- NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samples (NeurIPS 2024 · similarity 0.90)
- Right this way: Can VLMs Guide Us to See More to Answer Questions? (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
