Research map / Label-efficient and robust learning / Test-time adaptation
Unraveling Batch Normalization for Realistic Test-Time Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Diagnoses degraded test-time batch-norm adaptation with small batches as caused by reduced class diversity, and proposes TEMA (test-time EMA with adaptive momentum) plus a divergence-based layer-wise source/target statistic rectification, requiring no training or tuned hyperparameters.
Problem
Test-time adaptation via batch normalization degrades with realistic small test mini-batches because target batch statistics are inaccurate, largely due to reduced class diversity within the batch.
Approach
Attributes inaccurate target BN statistics to low class diversity in small batches and bridges the gap with a test-time exponential moving average whose momentum is chosen adaptively to balance class diversity against timeliness; a layer-wise rectification then blends source and target statistics per BN layer using a KL-based divergence, without training or tuning.
Key results
- On CIFAR-10-C under continual adaptation, the method averages 20.82% error across test batch sizes 200 to 1, versus 26.59% for the next-best method AdaptBN.
- On CIFAR-100-C the method averages 36.28% error and on ImageNet-C it averages 65.74% error across batch sizes.
- Error on CIFAR-10-C rises only from 20.20% at batch size 200 to 21.05% at batch size 1, whereas TENT and CoTTA degrade substantially at small batches.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add TEMA to current test-batch normalization statistics (ablation) | test-time BN adaptation · CIFAR-10-C, CIFAR-100-C, and ImageNet-C continual; Avg. across batch sizes | error rate: 37.38 -> 21.61; 56.06 -> 39.80; 82.89 -> 72.47 | Helped |
| add layer-wise rectification to TEMA (ablation) | TEMA · CIFAR-10-C, CIFAR-100-C, and ImageNet-C continual; Avg. across batch sizes | error rate: 21.61 -> 20.82; 39.80 -> 36.28; 72.47 -> 65.46 | Helped |
| use Ours instead of AdaptBN | continual adaptation · CIFAR-10-C, Avg. across batch sizes | error rate: 26.59 -> 20.82 | Helped |
| use Ours instead of AdaptBN | continual adaptation · CIFAR-100-C, Avg. across batch sizes | error rate: 37.55 -> 36.28 | Helped |
| use Ours instead of α-BN | continual adaptation · ImageNet-C, Avg. across batch sizes | error rate: 65.53 -> 65.74 | Hurt |
| use Ours instead of AdaptBN | mixed-domain adaptation · CIFAR-10-C mixed domain, Avg. across batch sizes | error rate: 35.91 -> 35.12 | Helped |
| use Ours instead of AdaptBN | gradual adaptation · CIFAR-10-C gradual, Avg. across batch sizes | error rate: 16.74 -> 13.91 | Helped |
Surprising findings
- Class diversity within a batch, rather than the raw sample count, is identified as the main driver of inaccurate target BN statistics under small batches.
Limitations
- The theoretical analysis assumes i.i.d. samples, which may not hold in all test streams.
- Early-stage performance is unsatisfactory because TEMA's target estimates are poor before enough data has arrived.
Experimental setup
Datasets: CIFAR-10-C CIFAR-100-C ImageNet-C
Benchmarks: CIFAR-10-C CIFAR-100-C ImageNet-C
Metrics: error rate
Baselines: TENT CoTTA SAR AdaContrast ETA TBN alpha-BN AdaptBN LAME
Builds on: TENT TBN alpha-BN AdaptBN
Closest papers by meaning
- Buffer layers for Test-Time Adaptation (NeurIPS 2025 · similarity 0.91)
- Towards Real-World Test-Time Adaptation: Tri-net Self-Training with Balanced Normalization (AAAI 2024 · similarity 0.89)
- Un-Mixing Test-Time Normalization Statistics: Combatting Label Temporal Correlation (ICLR 2024 · similarity 0.89)
- A Layer Selection Approach to Test Time Adaptation (AAAI 2025 · similarity 0.88)
- Backpropagation-Free Test-Time Adaptation via Probabilistic Gaussian Alignment (NeurIPS 2025 · similarity 0.88)
- SNAP: Low-Latency Test-Time Adaptation with Sparse Updates (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.