Research map / Vision-language and multimodal / Human and embodied modeling
CPathAgent: An Agent-based Foundation Model for Interpretable High-Resolution Pathology Image Analysis Mimicking Pathologists' Diagnostic Logic
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CPathAgent, an agent-based pathology foundation model that navigates high-resolution WSIs via zoom and move actions in a three-stage workflow, trained with a multi-stage strategy on CPathAgent-Instruct and evaluated on patch, huge-region (PathMMU-HR²) and WSI tasks.
Problem
Existing pathology foundation models either use MIL over patch features or single-pass LMMs that output final diagnoses without mimicking pathologists' multi-scale navigation or revealing reasoning, leaving them prone to shortcut learning and uninterpretable.
Approach
An agent that performs global screening on a WSI thumbnail, plans a navigation path of (x, y, magnification, focus) steps, then reasons over the sequence of cropped multi-scale views, with the data synthesized by Gemini-2.5-Pro guided by WSI reports and a three-stage progressive training of LLaVA-OneVision-style model with Qwen3-14B and CPath-CLIP.
Key results
- CPathAgent reaches 78.6% overall on PathMMU test (vs. CPath-Omni 72.2%, Gemini-2.5-Pro 67.5%).
- CPathAgent achieves 88.6% on PathMMU-HR² vs. 73.2% for Gemini-2.5-Pro and 71.7% for CPath-Omni.
- CPathAgent reaches 82.8% average balanced accuracy on six TCGA WSI classification tasks vs. 79.9% for ABMIL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train CPathAgent with the multi-stage agent-based workflow rather than use CPath-Omni | PathMMU test set · PathMMU test set, Tiny and All | accuracy (%): 72.4 -> 80.5; 72.2 -> 78.6 | Helped |
| Use CPathAgent instead of Gemini-2.5-Pro with agent prompting | PathMMU-HR² · PathMMU-HR², Overall | accuracy (%): 76.4 -> 88.6 | Helped |
| Apply CPathAgent's agent prompting strategy | Gemini-2.5-Pro · PathMMU-HR², Overall | accuracy (%): 73.2% to 76.4% | Helped |
| Apply CPathAgent's agent prompting strategy | GPT-4.1-mini · PathMMU-HR², Overall | accuracy (%): 60.1% to 62.3% | Helped |
| Use CPathAgent instead of ABMIL | TCGA WSI classification · TCGA WSI classification, Avg. | balanced accuracy (%): 79.9 -> 82.8 | Helped |
| Use CPathAgent instead of CPath-Omni | PathMMU-Pro · PathMMU-Pro, Test-Tiny and Test-All | accuracy (%): 62.1% versus 66.3%; 61.8% versus 65.3% | Helped |
| Add learned global screening (ablation) | CPathAgent · TCGA WSI classification, Avg. | balanced accuracy (%): 80.3 -> 82.8 | Helped |
| Add dynamic navigation and multi-scale reasoning (ablation) | CPathAgent · TCGA WSI classification, Avg. | balanced accuracy (%): 75.0 -> 82.8 | Helped |
| Apply agent-based inference prompting (ablation) | Gemini-2.5-Pro · TCGA WSI classification, Avg. | balanced accuracy (%): 72.1% to 65.4% | Hurt |
| Apply agent-based inference prompting (ablation) | Gemini-2.5-Pro · TCGA WSI classification | balanced accuracy (%): TCGA-BLCA: 59.8 -> 64.7; TCGA-THCA: 44.9 -> 46.3; TCGA-BRCA: 72.4 -> 52.0; TCGA-NSCLC: 89.2 -> 73.4; TCGA-RCC: 69.2 -> 5 | Mixed |
Surprising findings
- Applying CPathAgent's agent prompting directly to Gemini-2.5-Pro lowers WSI classification average accuracy from 72.1% to 65.4%.
- On the more rigorous PathMMU-Pro benchmark, expert performance (69.4%) exceeds CPathAgent (66.3%).
Limitations
- Training data is synthesized by Gemini-2.5-Pro and depends on closed-source model quality and WSI reports.
- WSI classification requires Gemini-2.5-Pro to map descriptions to labels, adding a non-agent step and gap to upper bound.
- OOD performance is not yet at clinical deployment standards; BLCA and THCA remain weak.
Experimental setup
Datasets: CPathAgent-Instruct CPath-PatchCaption CPath-PathInstruct HistGen reports TCGA WSIs PathMMU
Benchmarks: PathMMU PathMMU-HR² TCGA WSI classification (BRCA, NSCLC, RCC, ESCA, BLCA, THCA) CPTAC-Lung BRACS PathMMU-Pro
Metrics: accuracy balanced accuracy
Baselines: CPath-Omni Gemini-2.5-Pro GPT-4.1 GPT-4.1-mini PathGen-LLaVA Quilt-LLaVA ABMIL DSMIL
Builds on: LLaVA-OneVision CPath-Omni HistGen PathMMU
Closest papers by meaning
- CPath-Omni: A Unified Multimodal Foundation Model for Patch and Whole Slide Image Analysis in Computational Pathology (CVPR 2025 · similarity 0.90)
- PathVQ: Reforming Computational Pathology Foundation Model for Whole Slide Image Analysis via Vector Quantization (NeurIPS 2025 · similarity 0.88)
- PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images? (NeurIPS 2026 · similarity 0.88)
- Patho-R1: A Multimodal Reinforcement Learning-Based Pathology Expert Reasoner (AAAI 2026 · similarity 0.87)
- Patho-AgenticRAG: Towards Multimodal Agentic Retrieval-Augmented Generation for Pathology VLMs via Reinforcement Learning (AAAI 2026 · similarity 0.87)
- MLLM-HWSI: A Multimodal Large Language Model for Hierarchical Whole Slide Image Understanding (CVPR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.