Research map / Vision-language and multimodal / Multimodal LLMs

PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images?

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Instruction-tuned multimodal assistants · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces PathView-Bench, a deterministic, annotation-derived VQA benchmark (14 tasks, Region-FOV and Slide-FOV) to test multiscale fine-grained visual understanding of pathology MLLMs, evaluating 18 models and a LoRA-tuned Qwen3-VL-8B baseline.

Problem

Existing pathology MLLM benchmarks mainly score final diagnostic answers, captions, or reports, and rarely test whether models ground their answers in fine-grained visual content across multiple scales (local regions and whole slides).

Approach

Convert human-supervised labels and spatial annotations from 23 public pathology datasets into deterministic task targets across two fields of view (Region-FOV and Slide-FOV), including FOV-trap items whose correct answer is "insufficient information", and score programmatically without LLM-as-judge.

Key results

• Best frozen model reaches 0.201 Dice on Region-FOV object grounding (Qwen3-VL/GLM-4.6V-Flash range); best frozen counting MAE is 16.82 (GPT-5-mini).
• PathView-tuned Qwen3-VL-8B (LoRA) improves Region-FOV object grounding to 0.323 and spatial relation to 0.673, and Slide-FOV lesion grounding to 0.617 over the frozen models.
• Ordinary tasks show 12.4% average refusal, while safe-refusal rates are 30.9% for Region-FOV traps and 73.6% for Slide-FOV traps.

Surprising findings

• Correct high-level WSI QA answers do not reliably imply fine-grained localization or burden estimation; conditioned lesion grounding scores stay at 0.274 on matched slides.
• Medical-domain and pathology-specialized models do not consistently outperform general-purpose MLLMs.

Limitations

• Built from public datasets; not a complete clinical benchmark and does not assess full diagnostic reporting or clinical readiness.
• Excludes private longitudinal records, multi-timepoint workflows, and follow-up information.
• Slide-feature interfaces for pathology WSI models may lose pixel-level coordinate relationships, confounding spatial scores.

Experimental setup

Datasets: 23 public pathology imaging datasets (Zenodo, Kaggle, GitHub, challenge sites) SlideBench WSI-VQA

Benchmarks: PathView-Bench

Metrics: Dice accuracy F1 MAE entity recall SRR

Baselines: Random Choice Frequent Choice Gemini-2.5-Flash GPT-5-mini InternVL3.5 Lingshu HuatuoGPT-V Patho-R1 SlideChat WSI-LLaVA

Builds on: LoRA ms-swift

Closest papers by meaning

• PathFLIP: Fine-grained Language-Image Pretraining for Versatile Computational Pathology (AAAI 2026 · similarity 0.92)
• MLLM-HWSI: A Multimodal Large Language Model for Hierarchical Whole Slide Image Understanding (CVPR 2026 · similarity 0.91)
• CPath-Omni: A Unified Multimodal Foundation Model for Patch and Whole Slide Image Analysis in Computational Pathology (CVPR 2025 · similarity 0.91)
• PathVQ: Reforming Computational Pathology Foundation Model for Whole Slide Image Analysis via Vector Quantization (NeurIPS 2025 · similarity 0.91)
• PathGen-1.6M: 1.6 Million Pathology Image-text Pairs Generation through Multi-agent Collaboration (ICLR 2025 · similarity 0.90)
• SlideChat: A Large Vision-Language Assistant for Whole-Slide Pathology Image Understanding (CVPR 2025 · similarity 0.90)