Research map / Vision-language and multimodal / Multimodal LLMs

Dynamic Decision Learning: Test-Time Evolution for Abnormality Grounding in Rare Diseases

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Dynamic Decision Learning (DDL), a training-free test-time framework that optimizes prompts with an LLM optimizer and consolidates multi-view perturbed predictions via Hungarian matching to ground brain abnormalities with frozen Qwen2.5-VL models, yielding a consensus reliability score.

Problem

Abnormality grounding (bounding-box localization) in brain MRI for rare diseases with scarce labels, where single-pass LVLM predictions are unstable and unreliable under prompt and visual perturbations.

Approach

Treats grounding as adaptive test-time inference: an LLM optimizer evolves instructions against a small dev set (DAPE), and predictions across semantically-preserving image perturbations are aligned to a reference anchor via Hungarian matching (RHC), with a consensus reliability score combining recurrence and IoU consistency.

Key results

• On NOVA with the 7B model, DDL raises mAP@75 from 0.036 to 0.075 (+104.7%) over vanilla prompting.
• DDL outperforms SFT on NOVA at 3B (mAP@75 0.066 vs 0.046; mAP@25 0.298 vs 0.283) while LoRA gives only marginal gains.
• Reliability-score vs IoU correlation rises with scale on NOVA from r=0.096 (3B) to r=0.535 (72B).

Surprising findings

• Several prompting baselines such as chain-of-thought and visual-instruction methods substantially degrade localization relative to vanilla prompting.
• On BTD, the 3B model with DDL shows a slight drop in mAP@25 versus vanilla, and reliability scores are uncorrelated with IoU for 3B and 7B on BTD.

Limitations

• Increased test-time computation due to multiple views and prompt search.
• Reliability scores are poorly calibrated for small models (3B-7B) on easier tasks.
• Evaluated only on two brain MRI benchmarks and Qwen2.5-VL models.

Experimental setup

Datasets: BTD (brain tumor dataset, 293 held-out test samples) NOVA (281 rare pathologies, 806 held-out test samples)

Benchmarks: NOVA BTD

Metrics: mAP@25 mAP@50 mAP@75 IoU Pearson correlation mean absolute error

Baselines: Chain-of-thought Role-play Visual description Strict constraints VISER Visual-Instruct Meta-optimization Gradient-based optimization LoRA SFT

Builds on: Qwen2.5-VL Hungarian algorithm

Closest papers by meaning

• NOVA: A Benchmark for Rare Anomaly Localization and Clinical Reasoning in Brain MRI (NeurIPS 2025 · similarity 0.84)
• DDx-TRACE: A Benchmark for Medical Diagnostic Trajectories in VLMs (NeurIPS 2026 · similarity 0.84)
• Unscrambling disease progression at scale: fast inference of event permutations with optimal transport (NeurIPS 2024 · similarity 0.83)
• CMR-RD: Long-Tailed Adaptive VLM for Explainable CMR Diagnosis (CVPR 2026 · similarity 0.83)
• How Do Medical MLLMs Fail? A Study on Visual Grounding in Medical Images (ICLR 2026 · similarity 0.83)
• Adaptive Shrinkage Estimation for Personalized Deep Kernel Regression in Modeling Brain Trajectories (ICLR 2025 · similarity 0.83)