Research map / Vision-language and multimodal / Multimodal LLMs

EasyLens: A Training-Free Plug-and-Play Subtle-Lesion Representation Amplifier for Medical Vision-Language Models

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes EasyLens, a training-free, plug-and-play inference-time adapter that builds a pathology-anatomy prototype bank, selects lesion-relevant patch tokens via counterfactual prototype scoring, and residually amplifies them to improve subtle-lesion recognition in frozen medical VLMs.

Problem

Medical VLMs are insensitive to subtle lesions whose sparse, low-contrast visual evidence is diluted when local patch tokens are aggregated into global image embeddings; existing fixes require encoder pre-training or fine-tuning, limiting use with frozen VLMs.

Approach

Non-parametric prototype memory (lesion prototypes plus location-wise normal anatomy prototypes, support memories, morphology priors) enables counterfactual patch selection (lesion similarity minus normal consistency) and morphology-propagated residual enhancement of selected patch embeddings, with no weight updates.

Key results

• On ReX, EasyLens with MedGemma1.5 improves Stat./Sel./Gen. from 42.86/23.33/4.41 to 66.67/31.11/5.15.
• On LLaVA-Med, ReX Gen. rises from 3.93 to 32.37 with EasyLens.
• EasyLens achieves best results in all nine dataset-task combinations, with gains of 0.60 to 3.93 points over the strongest baseline.

Surprising findings

• A trainable MLP selector gets higher Stat. (69.39) than EasyTag, but lower Sel. and Gen.
• Removing the residual formulation increases Sel. (33.33 vs 31.11), slightly better than the full method on that metric.

Limitations

• Interface parameters (beta, k, s, m) are calibrated per backbone, so plug-and-play still requires some tuning.
• The subtle-lesion benchmark is constructed by the authors from grid-based pseudo-labels, with small sample sizes (e.g., 45-ish cases), limiting statistical reliability.
• Report generation lesion-token probe relies on a hand-defined lesion-token set.

Experimental setup

Datasets: ReXGroundingCT LIDC-IDRI AbdomenAtlas 3.0 Mini MIMIC-CXR Kvasir-SEG BKAI-Polyp

Benchmarks: ReX LIDC Abdomen MIMIC-CXR Kvasir-SEG BKAI-Polyp

Metrics: Stat. accuracy Sel. accuracy Gen. lesion-token probability BLEU-1 BLEU-4 METEOR ROUGE-L

Baselines: LLaVA-Med RadFM Lingshu Hulu-Med MedGemma MedKLIP KAD MLIP AdaMatch PLACE

Builds on: MedKLIP KAD MLIP AdaMatch PLACE

Closest papers by meaning

• Decomposing Disease Descriptions for Enhanced Pathology Detection: A Multi-Aspect Vision-Language Pre-training Framework (CVPR 2024 · similarity 0.86)
• Unleashing the Potential of Vision-Language Pre-Training for 3D Zero-Shot Lesion Segmentation via Mask-Attribute Alignment (ICLR 2025 · similarity 0.86)
• FOCUS: Knowledge-enhanced Adaptive Visual Compression for Few-shot Whole Slide Image Classification (CVPR 2025 · similarity 0.85)
• PathFLIP: Fine-grained Language-Image Pretraining for Versatile Computational Pathology (AAAI 2026 · similarity 0.85)
• BIOMEDICA: An Open Biomedical Image-Caption Archive, Dataset, and Vision-Language Models Derived from Scientific Literature (CVPR 2025 · similarity 0.85)
• Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understanding (ICLR 2025 · similarity 0.85)