Research map / Label-efficient and robust learning / Test-time adaptation
Buffer layers for Test-Time Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a plug-and-play "Buffer" layer (parallel 1x1/3x3 conv residual branch) that is updated during test-time adaptation while the pretrained backbone and BN layers stay frozen, improving BN-based TTA methods under small batch sizes and reducing forgetting.
Problem
Normalization-layer-based test-time adaptation (e.g., BN updates) is unstable under small test batch sizes and constrained by training-time statistics, and full-model updates are expensive and cause forgetting.
Approach
Insert a lightweight, randomly initialized convolutional Buffer module in parallel to the frozen pretrained backbone (residual addition with a learnable scale alpha, placed in early stages) and optimize only it with existing TTA objectives, instead of updating BN affine parameters.
Key results
- On CIFAR10-C (WRN28, BS=2), TENT@Buffer reduces error from 82.56% (TENT@BN) to 37.05%.
- On CIFAR10-W (BS=2), DeYo error falls from 84.37% (@BN) to 46.96% (@Buffer), and TENT from 89.30% to 64.14%.
- On ImageNet-C with ResNet50 (BS=2), TENT@Buffer reaches 93.14% error versus 96.29% for TENT@BN, with gains at small batch sizes but reduced or reversed gains at larger batch sizes for some methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace BN updates with Buffer updates | TENT · CIFAR10-C, WRN28, BS=2 | error rate (%): 82.56{ $\pm$0.40} -> 37.05{ $\pm$0.31} | Helped |
| Replace BN updates with Buffer updates | TTA methods including TENT, EATA, SAR, DeYo, CMF, and ROID · CIFAR10-C and CIFAR100-C, across architectures and batch sizes | error rate (%) | Mixed |
| Replace BN updates with Buffer updates | DeYo · CIFAR10-W, BS=2 | error rate (%): 84.37{\fontsize{4pt}{4pt}\selectfont$\pm$0.61} -> 46.96{\fontsize{4pt}{4pt}\selectfont$\pm$1.48} | Helped |
| Add joint BN and Buffer updates instead of Buffer-only updates | TTA methods on CIFAR10-W · CIFAR10-W, batch sizes 2, 4, and 16 | error rate (%) | Mixed |
| Replace BN updates with Buffer updates | CMF · ImageNet-C, ResNet50 (BN), BS=2 | error rate (%): 99.32{ $\pm$0.01} -> 93.24{ $\pm$0.03} | Helped |
| Use Buffer updates instead of BN updates | CMF · CIFAR10-C, continuous domains | average error rate (%): 28.20 -> 27.74 | Helped |
| Set alpha to 1e-1 rather than 1e-4 (ablation) | TENT @Buffer · CIFAR100-C, BS=2 | error rate (%): 97.88 -> 98.30 | Hurt |
| Set alpha to 1e-1 rather than 1e-3 (ablation) | TENT @Buffer · CIFAR100-C, BS=256 | error rate (%): 32.70 -> 30.96 | Helped |
| Use dual-path 1×1 and 3×3 convolutions (Module ④) rather than other tested designs (ablation) | TENT @Buffer · CIFAR10 and CIFAR100, ReLU placement | error rate (%): 29.36 -> 29.35; 20.28 -> 19.03; 48.29 -> 47.99; 34.45 -> 32.31 | Helped |
| Place Buffer only at early stages rather than middle or late stages (ablation) | TENT @Buffer · Small-batch adaptation | error rate (%) | Helped |
| Add Buffer at both early and middle stages (ablation) | TENT @Buffer · Large-batch adaptation | error rate (%) | Helped |
| Place Buffer at the final stage rather than earlier stages (ablation) | TENT @Buffer · CIFAR10-C and CIFAR100-C | error rate (%) | Hurt |
Surprising findings
- Buffer-only adaptation gains shrink or reverse at larger batch sizes (e.g., BS=16 on CIFAR10-W), where @BN often performs better, motivating joint BN+Buffer updates.
- On a mixed-domain ImageNet-C/CIFAR100-C setting, CMF@Buffer and DeYo@Buffer show large error drops at BS=2 while several methods show essentially no change.
Limitations
- Optimal Buffer architecture and placement are not identified; the reason placing it after activations works better is unclear.
- The scaling factor alpha is sensitive to initialization and has no principled initialization strategy.
- The adaptation objective is borrowed from existing TTA methods and not tailored to the Buffer layer; gains diminish at larger batch sizes when used without BN updates.
Experimental setup
Datasets: CIFAR10-C CIFAR100-C CIFAR10-W ImageNet-C
Benchmarks: CIFAR10-C CIFAR100-C CIFAR10-W ImageNet-C
Metrics: classification error rate (%)
Baselines: TENT EATA SAR DeYo CMF ROID RoTTA CoTTA AdaContrast EcoTTA L-TTA
Builds on: TENT EATA SAR DeYo ROID
Closest papers by meaning
- A Layer Selection Approach to Test Time Adaptation (AAAI 2025 · similarity 0.92)
- Unraveling Batch Normalization for Realistic Test-Time Adaptation (AAAI 2024 · similarity 0.91)
- Towards Real-World Test-Time Adaptation: Tri-net Self-Training with Balanced Normalization (AAAI 2024 · similarity 0.89)
- SNAP: Low-Latency Test-Time Adaptation with Sparse Updates (NeurIPS 2025 · similarity 0.88)
- AETTA: Label-Free Accuracy Estimation for Test-Time Adaptation (CVPR 2024 · similarity 0.87)
- SURGEON: Memory-Adaptive Fully Test-Time Adaptation via Dynamic Activation Sparsity (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.