Research map / Label-efficient and robust learning / Test-time adaptation
On Pitfalls of Test-Time Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Adapt with online TTT instead of using the baseline | CIFAR10-C evaluation · CIFAR10-C | error (%): 44.3 -> 20.0 ± 0.1 | Helped |
| Use episodic MEMO instead of the baseline | CIFAR10 label-shift evaluation · CIFAR10 label shift (α=0.01) | error (%): 7.8 ± 2.3 -> 0.1 ± 0.0 | Helped |
| Use online MEMO rather than episodic MEMO | MEMO · Common distribution shifts | average test error (%): 31.5% for episodic MEMO; 66.6% for online MEMO | Hurt |
| Use online rather than episodic adaptation | MEMO · CIFAR10-C continual shifts | error (%): 12.7 ± 0.1 (episodic); online result not reported | Mixed |
| Use TTA under strong label shift instead of the baseline | TTA methods other than MEMO · CIFAR10 label shift (α=0.01) | error (%): Baseline 7.8 ± 2.3; BN_adapt 77.8 ± 1.7 | Hurt |
| Increase the number of adaptation steps beyond the reported moderate setting (ablation) | Online SHOT · CIFAR10-C (Gaussian noise) | accuracy: 70.9% -> 68.4% | Hurt |
| Apply oracle model selection in online adaptation (ablation) | Online TTA · Streaming test batches | adaptation performance | Mixed |
| Use robust augmentation policies such as AugMix or PixMix during pretraining (ablation) | ResNet-26 with TTA · CIFAR10-C | adaptation performance | Mixed |
| Use a model with higher OOD quality (ablation) | TTA methods · CIFAR10-C | adaptation performance | Helped |
| Use a high-quality classifier with the feature extractor (ablation) | TTA methods · CIFAR10-C label-shift domains | error (%): SHOT attains a 5.4% error rate in test domain #0 | Mixed |
| Use online rather than episodic adaptation (ablation) | TTA methods · Common distribution shifts | error (%) | Mixed |
| Increase the learning rate or choose unsuitable adaptation hyperparameters (ablation) | TENT and SHOT · CIFAR10-C | accuracy: decrease of up to 59.2% for TENT and 64.4% for SHOT | Hurt |
Closest papers by meaning
- Evaluation of Test-Time Adaptation Under Computational Time Constraints (ICML 2024 · similarity 0.92)
- A Layer Selection Approach to Test Time Adaptation (AAAI 2025 · similarity 0.92)
- Unraveling Batch Normalization for Realistic Test-Time Adaptation (AAAI 2024 · similarity 0.90)
- SNAP: Low-Latency Test-Time Adaptation with Sparse Updates (NeurIPS 2025 · similarity 0.90)
- Buffer layers for Test-Time Adaptation (NeurIPS 2025 · similarity 0.90)
- Backpropagation-Free Test-Time Adaptation via Probabilistic Gaussian Alignment (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.