Research map / Label-efficient and robust learning / Noisy labels
Curriculum Fine-tuning of Vision Foundation Model for Medical Image Classification Under Label Noise
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CUFIT, a curriculum fine-tuning scheme for DINOv2 with Rein adapters that uses a linear probing module's agreement-based clean sample selection to train two adapter modules for medical image classification under symmetric label noise.
Problem
Fine-tuning vision foundation models for medical image classification degrades when training labels are noisy, and existing clean-sample selection methods train from scratch and do not exploit pre-trained VFM features.
Approach
Linear probing of a frozen VFM is robust to label noise, so a linear probing module selects clean samples via an agreement criterion to train an intermediate adapter, which in turn selects samples for a last adapter used at inference (LPM->IAM->LAM curriculum), with the three modules trained simultaneously and no noise-rate hyperparameter.
Key results
- At 40% symmetric noise, CUFIT reaches 79.1% on HAM10000 vs 74.1% for CoDis (the strongest baseline), and 98.3% on BloodMnist.
- On the real-world Kaggle-EyePACS-to-APTOS/FGADR setting, CUFIT reaches 64.4% total accuracy vs 63.9% for JoCor.
- Averaged across noise rates 10–60%, CUFIT achieves 78.3% on HAM10000 and 91.6% on OrgancMnist, above Co-teaching (75.5%, 87.9%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CUFIT curriculum fine-tuning instead of Co-teaching | DINOv2 ViT-small + Rein · HAM10000 mean (noise 0.1–0.6) | test accuracy (%): 75.5 -> 78.3 | Helped |
| Use CUFIT curriculum fine-tuning instead of CoDis | DINOv2 ViT-small + Rein · APTOS-2019 mean (noise 0.1–0.6) | test accuracy (%): 80.1 -> 81.6 | Helped |
| Use CUFIT curriculum fine-tuning instead of Co-teaching | DINOv2 ViT-small + Rein · BloodMnist mean (noise 0.1–0.6) | test accuracy (%): 94.7 -> 98.6 | Helped |
| Use CUFIT curriculum fine-tuning instead of JoCor | DINOv2 ViT-small + Rein · OrgancMnist mean (noise 0.1–0.6) | test accuracy (%): 88.0 -> 91.6 | Helped |
| Use CUFIT instead of Co-teaching | DINOv2 + Rein trained on Kaggle-EyePACS · APTOS-2019 test | test accuracy (%): 70.9 -> 69.8 | Hurt |
| Use CUFIT instead of JoCor | DINOv2 + Rein trained on Kaggle-EyePACS · FGADR test | test accuracy (%): 53.1 -> 53.7 | Helped |
| Use CUFIT instead of JoCor | DINOv2 + Rein trained on Kaggle-EyePACS · Combined APTOS-2019 + FGADR test | test accuracy (%): 63.9 -> 64.4 | Helped |
| Use CUFIT instead of the adapter-only method | Vision foundation models with adapters · Medical image classification with noisy labels | test accuracy | Helped |
| Use the IAM to select samples for the LAM rather than relying on the LPM alone | CUFIT · HAM10000 and APTOS-2019 with 40% noise | test accuracy | Helped |
Surprising findings
- Linear probing of the frozen DINOv2 outperforms the Rein adapter and full training at higher noise rates, showing adapters degrade under label noise.
- Full fine-tuning of DINOv2 collapses to around 56% on HAM10000 at 40% noise, below linear probing.
Limitations
- Evaluated mainly on simulated symmetric noise; real-world noise evaluated on a single setting.
- Relies on the assumption that clean labels form the majority within each noisy class.
- Does not compare against methods that modify the loss or architecture (e.g., semi-supervised learning).
Experimental setup
Datasets: HAM10000 APTOS-2019 BloodMnist OrgancMnist Kaggle-EyePACS FGADR CIFAR10 CIFAR100 ANIMAL10N ISIC 2018
Benchmarks: HAM10000 APTOS-2019 BloodMnist OrgancMnist Kaggle-EyePACS->APTOS-2019/FGADR CIFAR10 CIFAR100 ANIMAL10N
Metrics: test accuracy label precision label recall
Baselines: Full-training Linear probing Rein Co-teaching JoCor CoDis
Builds on: Co-teaching Co-teaching+ CoDis Rein DINOv2
Closest papers by meaning
- Systematic Comparison of Semi-supervised and Self-supervised Learning for Medical Image Classification (CVPR 2024 · similarity 0.85)
- Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks (ICLR 2024 · similarity 0.84)
- On the Role of Label Noise in the Feature Learning Process (ICML 2025 · similarity 0.84)
- NoiseGPT: Label Noise Detection and Rectification through Probability Curvature (NeurIPS 2024 · similarity 0.84)
- VFM-Adapter: Adapting Visual Foundation Models for Dense Prediction with Dynamic Hybrid Operation Mapping (AAAI 2025 · similarity 0.83)
- L2B: Learning to Bootstrap Robust Models for Combating Label Noise (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.