Research map / Video / Video-language understanding
OphCLIP: Hierarchical Retrieval-Augmented Learning for Ophthalmic Surgical Video-Language Pretraining
Ming Hu, Kun Yuan, Yaling Shen, Feilong Tang, Xiaohao Xu, Lin Zhou, Wei Li, Ying Chen, Zhongxing Xu, Zelin Peng, Siyuan Yan, Vinkle Srivastav and 8 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pretrain on OphVL dataset (ablation) | CLIP RN50 · Cat-21 zero-shot phase | accuracy: 13.3 -> 28.8 | Improved |
| pretrain on OphVL dataset (ablation) | CLIP RN50 · Cataract-1K zero-shot phase | accuracy: 6.1 -> 20.8 | Improved |
| add silent-video knowledge base and hierarchical retrieval (ablation) | OphCLIP (Caption prompt) · Cat-21 zero-shot phase | accuracy: 34.9 -> 39.9 | Improved |
| add silent-video knowledge base (ablation) | OphCLIP (Caption prompt) · Cataract-1K zero-shot phase | accuracy: 60.5 -> 61.3 | Improved |
| use Mix prompt (caption plus keywords) instead of Caption prompt (ablation) | OphCLIP with KB · Cat-21 zero-shot phase | accuracy: 39.9 -> 41.4 | Improved |
| use Mix prompt instead of Caption prompt (ablation) | OphCLIP without KB · Cataract-1K zero-shot phase | accuracy: 60.5 -> 55.9 | Hurt |
| add hierarchical retrieval-augmented pretraining over plain OphVL CLIP | CLIP* · Cataract-1K zero-shot phase | accuracy: 20.8 -> 62.8 | Improved |
| OphCLIP vs CLIP pretrained on OphVL | CLIP* · CaDIS-F zero-shot instrument | FPR: 85.1 -> 54.1 | Improved |
| OphCLIP vs CLIP pretrained on OphVL | CLIP* · Cat-21 linear probing 100% data | accuracy: 59.4 -> 72.1 | Improved |
| OphCLIP vs CLIP pretrained on OphVL | CLIP* · Cat-21 linear probing 10% data | accuracy: 47.6 -> 59.5 | Improved |
| domain pretraining for instrument recognition | CLIP · Cataract-1K zero-shot instrument | FPR: 100.0 -> 45.1 | Improved |
| OphCLIP vs CLIP on instrument recognition | CLIP · Cataract-1K zero-shot instrument | mAP: 13.9 -> 21.2 | Improved |
Closest papers by meaning
- Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentation (NeurIPS 2024 · similarity 0.87)
- Bridging Vision and Language for Robust Context-Aware Surgical Point Tracking: The VL-SurgPT Dataset and Benchmark (AAAI 2026 · similarity 0.86)
- OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM (CVPR 2024 · similarity 0.84)
- Towards Dynamic 3D Reconstruction of Hand-Instrument Interaction in Ophthalmic Surgery (NeurIPS 2025 · similarity 0.84)
- MedHorizon: Towards Long-context Medical Video Understanding in the Wild (NeurIPS 2026 · similarity 0.84)
- Multi-OphthaLingua: A Multilingual Benchmark for Assessing and Debiasing LLM Ophthalmological QA in LMICs (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
