Research map / 3D vision / 3D generation
Like an Ophthalmologist: Dynamic Selection Driven Multi-View Learning for Diabetic Retinopathy Grading
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SMVDR, a multi-view diabetic retinopathy grading model that uses lesion prompts from an OIR agent, token-level dynamic selection (MVTSM), and a multi-view mixture-of-experts fusion (MVMoEM) on a Mamba-Transformer backbone.
Problem
Diabetic retinopathy grading from multi-view fundus images, where single-view methods lose lesion information and redundant or conflicting view features hurt accuracy.
Approach
Mimics ophthalmologist reading by applying dynamic selection at multiple granularities: pixel-level lesion prompts, token pruning via Gumbel-softmax gating, and view-level sparse mixture-of-experts fusion, combined with a Mamba-Transformer backbone and multi-view interaction Mamba.
Key results
- SMVDR-M reaches 84.01% accuracy and 95.58% AUC on the MFIDDR multi-view dataset, outperforming the best multi-view baseline LFMVDR (82.15% accuracy, 94.81% AUC).
- Against the best single-view model (Vim), SMVDR-M and SMVDR-W improve accuracy by 4.69%-13.35%.
- Removing the multi-view mixture-of-experts module drops accuracy from 84.01% to 83.06%, and removing lesion prompts drops it to 81.64%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use mask prompts instead of weak prompts | SMVDR · MFIDDR test | accuracy: 83.03 -> 84.01 | Helped |
| add lesion prompts (ablation) | SMVDR-M · MFIDDR test | accuracy: 81.64 -> 84.01 | Helped |
| add MVTSM (ablation) | SMVDR-M · MFIDDR test | accuracy: 83.68 -> 84.01 | Helped |
| add MVIMamba (ablation) | SMVDR-M · MFIDDR test | accuracy: 83.87 -> 84.01 | Helped |
| add MVMoEM (ablation) | SMVDR-M · MFIDDR test | accuracy: 83.06 -> 84.01 | Helped |
| use the combined Mamba-Transformer backbone instead of Transformer blocks only (ablation) | SMVDR-M · MFIDDR test | accuracy: 83.17 -> 84.01 | Helped |
| add Mamba window blocks to Mamba global blocks and Transformer blocks (ablation) | SMVDR-M · MFIDDR test | accuracy: 81.96 -> 84.01 | Helped |
| add Mamba global blocks to Mamba window blocks and Transformer blocks (ablation) | SMVDR-M · MFIDDR test | accuracy: 83.40 -> 84.01 | Helped |
| use the combined Mamba-Transformer backbone instead of Mamba global and window blocks only (ablation) | SMVDR-M · MFIDDR test | accuracy: 83.22 -> 84.01 | Helped |
Surprising findings
- The Vim single-view baseline is strongest among single-view models, and the paper reports that per-view single-view models lag multi-view fusion substantially.
Limitations
- Evaluated on a single dataset (MFIDDR) only.
- Relies on an external lesion prompt generator (HACDR-Net) whose quality affects the prompts.
- Some per-grade sensitivity and F1 results are not best, e.g. Grade 4 recall is low.
Experimental setup
Datasets: MFIDDR
Benchmarks: MFIDDR
Metrics: accuracy precision sensitivity specificity kappa F1 AUC
Baselines: Inception ResNet V2 MobileNet V2 ResNet50 ResNeXt50 32x4d ConvNeXt-B Swin-B Vim RETFound MVCINN ETMC LFMVDR
Builds on: Vision Mamba HACDR-Net Vision Transformer focal loss sparse mixture of experts
Closest papers by meaning
- FOCUS: Benchmarking Retinal Model Generalization from Foundation Vision Encoders to Multimodal LLMs (NeurIPS 2026 · similarity 0.88)
- Domain Adaptive Diabetic Retinopathy Grading with Model Absence and Flowing Data (CVPR 2025 · similarity 0.86)
- HACDR-Net: Heterogeneous-Aware Convolutional Network for Diabetic Retinopathy Multi-Lesion Segmentation (AAAI 2024 · similarity 0.86)
- SIGraph: Saliency Image-Graph Network for Retinal Disease Classification in Fundus Image (AAAI 2025 · similarity 0.83)
- Deep Hierarchies and Invariant Disease-Indicative Feature Learning for Computer Aided Diagnosis of Multiple Fundus Diseases (AAAI 2025 · similarity 0.83)
- RLPeri: Accelerating Visual Perimetry Test with Reinforcement Learning and Convolutional Feature Extraction (AAAI 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.