Research map / Label-efficient and robust learning / OOD and anomaly detection
Masked Images Are Counterfactual Samples for Robust Fine-Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune with object-masked counterfactual images refilled from one other image | CLIP ViT-B/32 · OOD avg. (five OOD datasets) | average top-1 accuracy: 44.2 -> 53.1 | Helped |
| Fine-tune with object-masked counterfactual images refilled from one other image | CLIP ViT-B/16 · OOD avg. (five OOD datasets) | average top-1 accuracy: 52.8 -> 63.3 | Helped |
| Use object-mask rather than random-mask or context-mask (ablation) | Masked-image fine-tuning · OOD avg. (five OOD datasets) | average top-1 accuracy: 51.2 -> 53.1; 52.6 -> 53.1 | Mixed |
| Refill masked regions from one other image instead of dropping them (ablation) | Object-mask fine-tuning · OOD avg. (five OOD datasets) | average top-1 accuracy: 51.9 -> 53.1 | Helped |
| Refill masked regions from multiple images instead of dropping them (ablation) | Object-mask fine-tuning · OOD avg. (five OOD datasets) | average top-1 accuracy: 51.9 -> 52.6 | Helped |
| Use single-fill rather than multi-fill (ablation) | Object-mask fine-tuning · ImageNet | top-1 accuracy: 77.9 -> 77.5 | Hurt |
| Use single-fill rather than multi-fill (ablation) | Object-mask fine-tuning · OOD avg. (five OOD datasets) | average top-1 accuracy: 52.6 -> 53.1 | Helped |
| Apply random-mask with no-fill instead of no masking (ablation) | Masked-image fine-tuning · OOD avg. (five OOD datasets) | average top-1 accuracy: 51.3 -> 51.2 | No clear effect |
| Apply context-mask with no-fill instead of no masking (ablation) | Masked-image fine-tuning · OOD avg. (five OOD datasets) | average top-1 accuracy: 51.3 -> 51.1 | No clear effect |
| Apply object-mask with no-fill instead of no masking (ablation) | Masked-image fine-tuning · OOD avg. (five OOD datasets) | average top-1 accuracy: 51.3 -> 51.9 | Helped |
| Use WiSE-FT weight averaging with the proposed fine-tuned model (ablation) | Ours (single-fill) · ImageNet ID and OOD accuracy | top-1 accuracy | Mixed |
| Distill from a WiSE-FT teacher in addition to counterfactual feature distillation (ablation) | Ours · ImageNet ID and OOD accuracy | top-1 accuracy | Helped |
Closest papers by meaning
- Towards Calibrated Robust Fine-Tuning of Vision-Language Models (NeurIPS 2024 · similarity 0.85)
- Robust Fine-tuning of Zero-shot Models via Variance Reduction (NeurIPS 2024 · similarity 0.84)
- Tuning Stable Rank Shrinkage: Aiming at the Overlooked Structural Risk in Fine-tuning (CVPR 2024 · similarity 0.84)
- CRoFT: Robust Fine-Tuning with Concurrent Optimization for OOD Generalization and Open-Set OOD Detection (ICML 2024 · similarity 0.84)
- Directional Gradient Projection for Robust Fine-Tuning of Foundation Models (ICLR 2025 · similarity 0.84)
- Contrastive Tuning: A Little Help to Make Masked Autoencoders Forget (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.