Research map / Vision-language and multimodal / Human and embodied modeling
MEPNet: Medical Entity-Balanced Prompting Network for Brain CT Report Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MEPNet, which prompts a LLaMA3-8B decoder with per-entity visual embeddings (knowledge-driven joint attention) and learning-status words so it balances learning across brain CT medical entities for report generation.
Problem
Brain CT report generation suffers from biased learning across many medical entities (anatomy regions and lesions) with inconsistent 3D spatial patterns, causing repetitive and inaccurate reports.
Approach
Use visual embeddings of individual medical entities (distilled via knowledge-driven joint attention over explicit graph and implicit learned adjacency) plus learning-status words derived from per-entity classification losses as multi-modal prompts for an LLM decoder, instead of classified entity labels.
Key results
- On BCT-CHR, MEPNet reaches CE F1 52.0 and CIDEr 22.3, versus 51.4 F1 and 22.2 CIDEr for the best prior WGAM-HI.
- On CTRG-Brain, MEPNet achieves BLEU-4 34.4 and CIDEr 86.6, above HILT (33.3 and 80.0).
- Ablation on BCT-CHR shows entity visual embeddings with both explicit and implicit adjacency raise F1 from 33.7 to 44.0, and adding status embeddings raises it to 51.4 (ablation table).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MEPNet rather than the strongest listed baseline | Brain CT report generation · CTRG-Brain | CIDEr: 50.3 -> 86.6 | Helped |
| Use MEPNet rather than the strongest listed baseline | Brain CT report generation · BCT-CHR | Recall: 66.8 -> 68.9 | Helped |
| Use MEPNet rather than the strongest listed baseline | Brain CT report generation · BCT-CHR | BLEU-4: 15.6 -> 14.7 | Hurt |
| Add entity visual prompts and both explicit and predicted adjacency matrices (ablation) | MEPNet · BCT-CHR | F1: 33.7 -> 44.0 | Helped |
| Add status prompts to entity visual prompts and both adjacency matrices (ablation) | MEPNet · BCT-CHR | F1: 44.0 -> 51.4 | Helped |
| Use predicted adjacency M_I rather than expert adjacency M_E with entity visual prompts (ablation) | MEPNet · BCT-CHR | F1: 35.4 -> 43.9 | Helped |
| Add classified entity category prompts to entity visual prompts (ablation) | MEPNet · BCT-CHR | CIDEr: 21.3 -> 18.9 | Hurt |
| Add classified entity category prompts to entity visual and status prompts (ablation) | MEPNet · BCT-CHR | F1: 51.4 -> 46.6 | Hurt |
Surprising findings
- Adding classified entity categories to the prompt lowered scores, the opposite of what prior 2D MRG work reports.
- MEPNet has lower BLEU-4 on BCT-CHR than WGAM-HI and lower precision than some ablations, attributed to the status embeddings broadening entity coverage.
Limitations
- Lower BLEU-4 on BCT-CHR and lower precision than some ablation variants.
- Relies on a hand-built brain pathological knowledge graph and a classifier whose quality bounds performance.
Experimental setup
Datasets: BCT-CHR CTRG-Brain CQ500
Benchmarks: BCT-CHR CTRG-Brain
Metrics: BLEU-4 METEOR ROUGE-L CIDEr clinical F1 clinical recall clinical precision
Baselines: Up-Down WCL WGAM PGCA WGAM-HI PromptMRG HILT
Builds on: LoRA PromptMRG HILT WGAM PGCA
Closest papers by meaning
- Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understanding (ICLR 2025 · similarity 0.86)
- AHIVE: Anatomy-aware Hierarchical Vision Encoding for Interactive Radiology Report Retrieval (CVPR 2024 · similarity 0.85)
- Instance-level Expert Knowledge and Aggregate Discriminative Attention for Radiology Report Generation (CVPR 2024 · similarity 0.85)
- Toward a Vision-Language Foundation Model for Medical Data: Multimodal Dataset and Benchmarks for Vietnamese PET/CT Report Generation (NeurIPS 2025 · similarity 0.85)
- Visual and Domain Knowledge for Professional-level Graph-of-Thought Medical Reasoning (ICML 2025 · similarity 0.85)
- OmniBrainBench: A Comprehensive Multimodal Benchmark for Brain Imaging Analysis Across Multi-stage Clinical Tasks (CVPR 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.