Research map / Vision-language and multimodal / CLIP and zero-shot
From Panel to Pixel: Zoom-In Vision-Language Pretraining from Biomedical Scientific Literature
Kun Yuan, Min Sun, Zhen Chen, Alejandro Lozano, Xiangteng He, Shi Li, Nassir Navab, Xiaoxiao Sun, Nicolas Padoy, Serena Yeung-Levy
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multi-granularity (figure, panel, region) supervision from Panel2Patch | BMC-CLIP ViT-L/14 · Single-panel short-context retrieval, I2T R@10 | R@10 (%): 73.53 -> 79.90 | Improved |
| add region-level supervision to single-panel data (full Panel2Patch vs single-panel only) (ablation) | Panel2Patch · Bounding-box to text retrieval, I2T R@5 | R@5 (%): 17.65 -> 20.24 | Improved |
| add single-panel data to region-only training (ablation) | Panel2Patch · Single-panel short-context retrieval, I2T R@5 | R@5 (%): 61.60 -> 68.14 | Improved |
| add alternating coarse-to-fine training instead of a single level only (ablation) | Panel2Patch · Multi-panel retrieval, 100K pairs, I2T R@5 | R@5 (%): 21.54 -> 22.07 | Improved |
| alternating training vs single-panel-only on single-panel test (ablation) | Panel2Patch · Single-panel retrieval, 100K pairs, I2T R@10 | R@10 (%): 15.36 -> 14.79 | No clear effect |
| train on multi-panel only then test on single-panel (no alternating) (ablation) | Panel2Patch · Cross-level retrieval, I2T R@10 | R@10 (%): Multi->Single 8.14 vs Multi->Multi 29.35 | Hurt |
| train on single-panel only then test on multi-panel (no alternating) (ablation) | Panel2Patch · Cross-level retrieval, I2T R@10 | R@10 (%): Single->Multi 12.37 vs Ours->Multi 29.59 | Hurt |
| train on 400K hierarchical pairs instead of much larger corpora | BMC-CLIP ViT-L/14 · Zero-shot classification, 6 specialties (avg) | accuracy (avg): 47.85 -> 50.25 | Improved |
| Panel2Patch vs BMC-LongCLIP on Ophthalmology zero-shot | BMC-CLIP ViT-L/14 · Zero-shot classification (Ophthalmology) | accuracy: 59.80 -> 35.78 | Hurt |
| Panel2Patch on bounding-box text retrieval vs BMC-CLIP at higher recall | BMC-CLIP ViT-L/14 · Bounding-box to text retrieval, I2T R@5 | R@5 (%): 20.98 -> 20.24 | Mixed |
Closest papers by meaning
- BIOMEDICA: An Open Biomedical Image-Caption Archive, Dataset, and Vision-Language Models Derived from Scientific Literature (CVPR 2025 · similarity 0.88)
- LLaDA-MedV: Exploring Large Language Diffusion Models for Biomedical Image Understanding (CVPR 2026 · similarity 0.87)
- SlideChat: A Large Vision-Language Assistant for Whole-Slide Pathology Image Understanding (CVPR 2025 · similarity 0.87)
- Enhancing Medical Large Vision-Language Models via Alignment Distillation (AAAI 2026 · similarity 0.87)
- PathFLIP: Fine-grained Language-Image Pretraining for Versatile Computational Pathology (AAAI 2026 · similarity 0.87)
- Few-Shot Learning from Gigapixel Images via Hierarchical Vision-Language Alignment and Modeling (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
