Research map / Adversarial robustness and security / Certified and empirical robustness
Distilling Out-of-Distribution Robustness from Vision-Language Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Distill CLIP-ViT-L/14 using teacher-generated discretized adversarial examples | ViT-B/16 · ImageNet-Rendition | top-1 accuracy: 47.3 -> 65.1 | Helped |
| Distill CLIP-ViT-L/14 using teacher-generated discretized adversarial examples | ViT-B/16 · ImageNet-C | mCE: 44.7 -> 53.2 | Hurt |
| Distill CLIP-ViT-L/14 using teacher-generated discretized adversarial examples | ResNet50 · ImageNet-C | mCE: 74.2 -> 67.4 | Helped |
| Combine DAD with DAT training on the student's adversarial samples (ablation) | ViT-B/16 · ImageNet | top-1 accuracy: 79.6 -> 81.9 | Helped |
| Combine DAD with DAT training on the student's adversarial samples (ablation) | ViT-B/16 · ImageNet-Rendition | top-1 accuracy: 65.1 -> 53.2 | Hurt |
| Replace VQGAN with Stable Diffusion as the DAD discretizer (ablation) | ViT-B/16 DAD · ImageNet-Rendition | top-1 accuracy: 65.1 -> 45.9 | Hurt |
| Remove gradient-based perturbation optimization and sample VQGAN discretizations (ablation) | ViT-B/16 DAD · ImageNet-Sketch | top-1 accuracy: 46.1 -> 34.9 | Hurt |
| Remove distillation terms and train on DAD examples as ordinary augmentation (ablation) | ViT-B/16 DAD · ImageNet-Rendition | top-1 accuracy: 65.1 -> 51.7 | Hurt |
| Distill from CLIP-ViT-L/14 rather than CLIP-RN101 (ablation) | ViT-B/16 with DAD · ImageNet-Rendition | top-1 accuracy: 48.7 -> 65.1 | Helped |
| Distill CLIP-ViT-L/14 using the original KD objective on clean data (ablation) | ResNet34 · ImageNet-Rendition | top-1 accuracy: 32.4 -> 41.2 | Helped |
Closest papers by meaning
- Out of Thin Air: Exploring Data-Free Adversarial Robustness Distillation (AAAI 2024 · similarity 0.90)
- Learning Better Certified Models from Empirically-Robust Teachers (NeurIPS 2026 · similarity 0.89)
- Robust Distillation via Untargeted and Targeted Intermediate Adversarial Samples (CVPR 2024 · similarity 0.89)
- Towards Adversarially Robust Dataset Distillation by Curvature Regularization (AAAI 2025 · similarity 0.88)
- Impartial Adversarial Distillation: Addressing Biased Data-Free Knowledge Distillation via Adaptive Constrained Optimization (AAAI 2024 · similarity 0.88)
- Gain from Neighbors: Boosting Model Robustness in the Wild via Adversarial Perturbations Toward Neighboring Classes (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.