Research map / Vision-language and multimodal / Multimodal LLMs

Uni-Hema: Unified Model for Digital Hematopathology

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Uni-Hema, a unified vision-language model that jointly handles detection, classification, segmentation, morphology prediction, VQA and masked language modeling across 46 hematology datasets, using a Hema-Former fusion module.

Problem

Digital hematopathology tasks (detection, classification, segmentation, morphology reasoning) across leukemia, malaria, anemia and sickle cell disease are handled by separate single-task, single-dataset models, which limits scalability and clinical applicability.

Approach

A multi-stage trained unified architecture (ResNet-50 + DINO-DETR-style encoder/decoder + T5 text encoder/decoder) with a Hema-Former module (cross-modal fusion, text-guided refinement, single-cell feature extractor, query-guided mask former) that routes visual and textual features to task-specific heads, with semi-synthetic VQA and MLM datasets generated from morphology annotations.

Key results

• Detection mean mAP50 of 61.1 for Uni-Hema vs 60.4 for DINO and 59.4 for YOLO; best on 4 leukemia FoV datasets (e.g., H_40x_C2 43.6 vs 40.6 AttriDet).
• Segmentation mean Dice 91.7 for Uni-Hema vs 93.7 for TransNetR (TransNetR remains best on mean), while Uni-Hema is 99.9 on Elsafty.
• Unseen-dataset single-cell classification mean F1 90.8 vs 89.9 for DinoBloom-S; VQA BLEU-4 56.4 and V-MLM BLEU-4 79.8.

Surprising findings

• Uni-Hema trails single-task baselines on several detection datasets such as Sickle Cell (67.0 vs 73.6 DINO) and Parasites (36.2 vs 46.1 YOLO), which the authors attribute to limited training samples.
• The segmentation mean Dice (91.7) is below TransNetR (93.7) on the reported datasets despite the unified training.

Limitations

• Trails specialized single-task models on some detection datasets with few samples (sickle cell, parasites).
• Segmentation uses a simple upsampler due to resource constraints, which may reduce edge sharpness.
• VQA and MLM datasets are semi-synthetic, generated by BioMistral and Gemini 1.5, and not intended for medical use.

Experimental setup

Datasets: H_40x_C2 H_100x_C2 L_40x_C2 L_100x_C2 H_1000x L_1000x H_400x L_400x Sickle Cell Parasites BCCD TXL

Benchmarks: H_40x_C2 H_100x_C2 L_40x_C2 L_100x_C2 H_1000x L_1000x H_400x L_400x Sickle Cell Parasites BCCD TXL

Metrics: mAP50 F1 Dice score BLEU-4 F1-macro

Baselines: DINO YOLO U-Net NanoNet TransNetR DinoBloom-S DINOv2-S DINOv3-S ResNet-50 AttriDet

Builds on: DINO-DETR Mask DINO T5 ResNet

Closest papers by meaning

• PathVQ: Reforming Computational Pathology Foundation Model for Whole Slide Image Analysis via Vector Quantization (NeurIPS 2025 · similarity 0.83)
• MLLM-HWSI: A Multimodal Large Language Model for Hierarchical Whole Slide Image Understanding (CVPR 2026 · similarity 0.83)
• PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images? (NeurIPS 2026 · similarity 0.83)
• BIOMEDICA: An Open Biomedical Image-Caption Archive, Dataset, and Vision-Language Models Derived from Scientific Literature (CVPR 2025 · similarity 0.82)
• Patho-R1: A Multimodal Reinforcement Learning-Based Pathology Expert Reasoner (AAAI 2026 · similarity 0.82)
• FOCUS: Knowledge-enhanced Adaptive Visual Compression for Few-shot Whole Slide Image Classification (CVPR 2025 · similarity 0.82)