Research map / Continual learning / Domain and prompt-based
Continual-MAE: Adaptive Distribution Masked Autoencoders for Continual Test-Time Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ADMA, a continual test-time adaptation method that combines distribution-aware masked autoencoding (MC-dropout uncertainty-based masking with consistency loss) and HOG feature reconstruction on top of ViDA, evaluated on CIFAR10C/CIFAR100C/ImageNet-C classification and Cityscapes-to-ACDC segmentation.
Problem
Continual test-time adaptation of a source-pretrained model to continually shifting unlabeled target distributions suffers from miscalibrated predictions, noisy pseudo-labels, error accumulation and catastrophic forgetting under entropy minimization or teacher-student pseudo-labeling.
Approach
Replace entropy minimization and mean-teacher pseudo-labeling with masked image modeling: select masked tokens by MC-dropout token-wise uncertainty (Distribution-aware Masking), enforce a consistency loss between masked and unmasked predictions, and reconstruct HOG descriptors of masked patches, which are argued to be invariant to domain shift.
Key results
- On CIFAR10-to-CIFAR10C, mean error drops from 28.2% (source) to 12.6%, versus 20.7% for ViDA.
- On Cityscapes-to-ACDC, mIoU reaches 61.8%, +5.1 over source and +3.2 over CoTTA.
- On ImageNet-to-ImageNet-C, mean error is 42.5% versus 55.8% for source and 50.0% for VDP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ADMA instead of ViDA | ViT-base · CIFAR10-to-CIFAR10C | classification error rate (%): 20.7 -> 12.6 | Helped |
| use ADMA instead of ViDA | ViT-base · CIFAR100-to-CIFAR100C | classification error rate (%): 27.3 -> 26.4 | Helped |
| use ADMA instead of VDP | ViT-base · ImageNet-to-ImageNet-C | classification error rate (%): 50.0 -> 42.5 | Helped |
| use ADMA instead of SVDP | Segformer-B5 · Cityscapes-to-ACDC | mIoU: 61.3 -> 61.8 | Helped |
| add random masking and consistency constraints (ablation) | source model · CIFAR10-to-CIFAR10C | classification error rate (%): 28.2 -> 17.1 | Helped |
| replace random masking with Distribution-aware Masking (ablation) | random-masking adaptation · CIFAR10-to-CIFAR10C | classification error rate (%): 17.1 -> 14.4 | Helped |
| add HOG reconstruction to random masking (ablation) | random-masking adaptation · CIFAR10-to-CIFAR10C | classification error rate (%): 17.1 -> 15.8 | Helped |
| add HOG reconstruction to DaM (ablation) | DaM adaptation · CIFAR10-to-CIFAR10C | classification error rate (%): 14.4 -> 12.6 | Helped |
| use RGB pixels instead of HOG as the reconstruction target (ablation) | ADMA · ImageNet-to-ImageNet-C | classification error rate (%): 43.6 -> 49.2 | Hurt |
| use CLIP features instead of HOG as the reconstruction target (ablation) | ADMA · ImageNet-to-ImageNet-C | classification error rate (%): 43.6 -> 48.5 | Hurt |
| use Mean-Teacher features instead of HOG as the reconstruction target (ablation) | ADMA · ImageNet-to-ImageNet-C | classification error rate (%): 43.6 -> 48.1 | Hurt |
| use SIFT features instead of HOG as the reconstruction target (ablation) | ADMA · ImageNet-to-ImageNet-C | classification error rate (%): 43.6 -> 50.4 | Hurt |
Surprising findings
- Reconstructing a CLIP feature target gave less gain than HOG, and reconstructing Mean-Teacher features was only slightly better than CLIP, suggesting high-level features still carry domain shift.
Limitations
- Gains on some corruption types are smaller; the authors note DaM alone may show limited robustness in continual adaptation.
- Method depends on injecting ViDA adapters, so standalone contribution is entangled with the prior method.
- Uncertainty estimation and HOG pipeline add design hyperparameters (mask ratio, lambda=0.5, m=10) selected without a sensitivity study in the text.
Experimental setup
Datasets: CIFAR10 CIFAR10C CIFAR100 CIFAR100C ImageNet ImageNet-C Cityscapes ACDC
Benchmarks: CIFAR10-to-CIFAR10C CIFAR100-to-CIFAR100C ImageNet-to-ImageNet-C Cityscapes-to-ACDC
Metrics: error rate mIoU
Baselines: Source Pseudo-label TENT CoTTA VDP ViDA SVDP
Builds on: Masked Autoencoders ViDA MIC Tent CoTTA
Closest papers by meaning
- A Versatile Framework for Continual Test-Time Domain Adaptation: Balancing Discriminability and Generalizability (CVPR 2024 · similarity 0.89)
- Ranked Entropy Minimization for Continual Test-Time Adaptation (ICML 2025 · similarity 0.87)
- PALM: Pushing Adaptive Learning Rate Mechanisms for Continual Test-Time Adaptation (AAAI 2025 · similarity 0.86)
- ViDA: Homeostatic Visual Domain Adapter for Continual Test Time Adaptation (ICLR 2024 · similarity 0.86)
- DPCore: Dynamic Prompt Coreset for Continual Test-Time Adaptation (ICML 2025 · similarity 0.86)
- Test-time Adaptation in Non-stationary Environments via Adaptive Representation Alignment (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.