Research map / Label-efficient and robust learning / Self-supervised and contrastive
Denoising Masked Autoencoders Help Robust Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace MAE pre-training with DMAE (adding a denoising objective) for linear probing (ablation) | ViT-B encoder · ImageNet linear probing, sigma=0.25 | certified accuracy at r=0.5: +32.0 (45.3 vs 13.3) | Helped |
| Denoising pre-training (DMAE) instead of MAE, fine-tuned with RS (ablation) | ViT-B fine-tuning · ImageNet sigma=0.5 | certified accuracy at r=1.0: +1.7 (53.2 vs 51.5) | Helped |
| Denoising pre-training (DMAE) instead of MAE, fine-tuned with CR (ablation) | ViT-B fine-tuning · ImageNet sigma=0.5 | certified accuracy at r=1.0: +2.0 (57.9 vs 55.9) | Helped |
| Longer pre-training (1100 vs 700 epochs) (ablation) | DMAE ViT-B · ImageNet sigma=0.5 | certified accuracy at r=1.0: +2.4 (57.9 vs 55.5) | Helped |
| Consistency regularization fine-tuning instead of RS (ablation) | DMAE ViT-L · ImageNet sigma=0.5 | certified accuracy at r=1.0: +5.3 (64.6 vs 59.3) | Helped |
| Consistency regularization fine-tuning instead of RS (ablation) | DMAE ViT-L · ImageNet sigma=1.0 | certified accuracy at r=3.0: +5.7 (27.5 vs 21.8) | Helped |
| Continued pre-training on CIFAR-10 before fine-tuning (ablation) | DMAE ViT-B on CIFAR-10 · CIFAR-10 sigma best | certified accuracy at r=0.5: +2.5 (67.1 vs 64.6) | Helped |
| Use DMAE ViT-B (1/10 params) instead of diffusion+BEiT two-stage pipeline | ImageNet certified accuracy · ImageNet sigma=0.5 | certified accuracy at r=1.5: DMAE ViT-B 47.8 vs Diffusion+BEiT 38.1 at r=1.5 (better at r>=1.0); worse at r=0.5 (69.6 vs 71.1 at sigma 0.25) | Mixed |
| Scale to DMAE ViT-L (304M) with CR fine-tuning | ImageNet · ImageNet sigma=1.0 | certified accuracy at r=2.0: 41.5 vs 29.5 for Diffusion+BEiT | Helped |
Closest papers by meaning
- Masked Autoencoders Are Scalable Vision Learners (CVPR 2022 · similarity 0.89)
- Self-Guided Masked Autoencoder (NeurIPS 2024 · similarity 0.87)
- Ditch the Denoiser: Emergence of Noise Robustness in Self-Supervised Learning from Data Curriculum (NeurIPS 2025 · similarity 0.86)
- Contrastive Tuning: A Little Help to Make Masked Autoencoders Forget (AAAI 2024 · similarity 0.86)
- Learning Mask Invariant Mutual Information for Masked Image Modeling (ICLR 2025 · similarity 0.85)
- Deconstructing Denoising Diffusion Models for Self-Supervised Learning (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.