Research map / Vision-language and multimodal / Multimodal LLMs

Criterion-Conditional In-Context Learning: Evaluating Criterion-Shift Adaptation in Vision-Language Models

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Criterion-Conditional In-Context Learning (CC-ICL), metrics (Criterion Invariance/Sensitivity), and CC-Bench, finding rigid boundary bias in VLMs and showing multi-criterion LoRA training (MCT) substantially improves criterion sensitivity of 7B models.

Problem

Standard multimodal in-context learning evaluations assume a fixed decision criterion once the task is induced, so they cannot measure whether VLMs adapt decision boundaries to shifting criteria under fixed task semantics.

Approach

Treat the decision criterion as a latent variable inferred from the support set, evaluate each query under multiple support sets that differ only in the label of criterion-sensitive samples, and score models with paired metrics (CS for adaptation, CI for robustness) on a dual-level category/sub-category benchmark; then train with multi-criterion LoRA supervision.

Key results

• Base VLMs show rigid boundary bias: most models score below 20% Criterion Sensitivity on CC-Bench, with Qwen2.5-VL-7B reaching 0.00% CS on Video Surveillance.
• MCT on Qwen2.5-VL-7B raises average overall accuracy by +44.00 (Industrial), +46.30 (Medical), and +49.52 (Video) points, with CS gains of +54.53, +74.57, and +45.71.
• Cross-domain MCT transfers to unseen domains, e.g., Industrial-trained model raises Medical CS from 8.54% to 57.13%, while MMBench/MMStar remain stable.

Surprising findings

• Text-only prompting increases invariance but does not improve criterion sensitivity, reinforcing rigid boundaries.
• Increasing shots from 3 to 9 yields limited and inconsistent CS gains for open-source models.

Limitations

• Transfer from static image domains to video surveillance remains nearly absent, indicating modality gaps.
• Evaluation restricted to binary yes/no labels and three domains with a fixed 3-shot protocol.

Experimental setup

Datasets: MVTec AD RadImageNet-VQA UCF-Crime NWPU Campus MMBench MMStar

Benchmarks: CC-Bench MMBench MMStar

Metrics: Criterion Sensitivity Criterion Invariance accuracy

Baselines: GPT-4o Gemini 2.5 Pro Qwen2.5-VL-7B-Instruct Qwen2.5-VL-32B-Instruct Qwen3-VL-8B-Instruct InternVL3.5-8B LLaVA-OneVision-1.5-8B-Instruct

Builds on: LoRA MVTec AD RadImageNet-VQA UCF-Crime

Closest papers by meaning

• In-Context Learning Can Help Vision Language Models Overcome Training Prior (NeurIPS 2026 · similarity 0.89)
• A Closer Look at the Few-Shot Adaptation of Large Vision-Language Models (CVPR 2024 · similarity 0.88)
• Capturing In-Context Learning Dynamics with Task Operators (NeurIPS 2026 · similarity 0.88)
• Large (Vision) Language Models are Unsupervised In-Context Learners (ICLR 2025 · similarity 0.88)
• Dual Memory Networks: A Versatile Adaptation Approach for Vision-Language Models (CVPR 2024 · similarity 0.88)
• The Illusion of Progress? A Critical Look at Test-Time Adaptation for Vision-Language Models (NeurIPS 2025 · similarity 0.88)