Research map / Label-efficient and robust learning / Test-time adaptation
Effortless Active Labeling for Long-Term Test-Time Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EATTA, an active test-time adaptation method that annotates at most one sample per batch (or per multiple batches), selecting border samples via feature-perturbation sensitivity and balancing supervised and unsupervised losses with gradient-norm-based dynamic weights.
Problem
Long-term test-time adaptation suffers from error accumulation, and active TTA methods require annotations that grow with batch count, making annotation costs prohibitive.
Approach
Select the single most valuable sample per batch as the one lying at the border between source and target distributions, identified by prediction-confidence sensitivity to feature noise on its pseudo-label; then balance supervised and unsupervised loss gradients using gradient-norm-based dynamic weights smoothed by EMA.
Key results
- On ImageNet-C under CTTA with one annotation per batch, EATTA reduces average error to 53.8% vs. 60.6% for the ATTA baseline and 56.6% for SimATTA (buffer 300) per the Table 5 comparison.
- EATTA with one annotation per five batches still outperforms the one-per-batch baseline on ImageNet-C, per Table 6.
- On PACS CTTA, EATTA with one annotation per batch achieves 77.9% average accuracy vs. 69.6% for the baseline in the first sequence set.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EATTA with three labels per batch and a buffer of 300 | ResNet-50 BN test-time adaptation · ImageNet-C CTTA | Avg. Err.: 53.7 -> 50.9 | Helped |
| Use EATTA with three labels per batch and a buffer of 300 | ResNet-50 BN test-time adaptation · ImageNet-C FTTA | Avg. Err.: 52.0 -> 50.7 | Helped |
| Use EATTA with three labels per batch and a buffer of 300 | ImageNet test-time adaptation · ImageNet-R, ImageNet-K, and ImageNet-A FTTA | Avg. Err.: 47.8 -> 45.7; 59.5 -> 57.9; 83.5 -> 81.8 | Helped |
| Add prediction difference-based sample selection (ablation) | EATTA · ImageNet-C | Avg. Err.: 54.7 -> 53.8 | Helped |
| Add class balancing (ablation) | EATTA with prediction difference selection · ImageNet-C | Avg. Err.: 58.6 -> 57.3 | Helped |
| Add gradient norm-based debiasing (ablation) | EATTA with prediction difference selection · ImageNet-C | Avg. Err.: 59.6 -> 55.2 | Helped |
| Add EMA refinement to gradient norm-based debiasing (ablation) | EATTA with prediction difference selection and gradient norm debiasing · ImageNet-C | Avg. Err.: 55.2 -> 54.3 | Helped |
| Add gradient norm-based debiasing and EMA to prediction difference selection and class balancing (ablation) | EATTA · ImageNet-C | Avg. Err.: 57.3 -> 53.8 | Helped |
| Add prediction difference-based sample selection (ablation) | EATTA with gradient norm debiasing and EMA · ImageNet-C | Avg. Err.: 54.7 -> 54.3 | Helped |
| Reduce annotation frequency to one sample every three or five batches (ablation) | EATTA · ImageNet-C, ImageNet-R, ImageNet-K, and ImageNet-A | Avg. Err.: 66.9 -> 68.6; 66.9 -> 69.5 | Hurt |
| Replace prediction difference sample selection with maximum entropy selection (ablation) | EATTA · ImageNet-C, ImageNet-R, ImageNet-K, and ImageNet-A | Avg. Err.: 66.9 -> 69.8 | Hurt |
| Replace prediction difference sample selection with random selection (ablation) | EATTA · ImageNet-C, ImageNet-R, ImageNet-K, and ImageNet-A | Avg. Err.: 66.9 -> 68.0 | Hurt |
Surprising findings
- Uncertainty-based active learning selection (max entropy, least confidence, min margin) performs worse than random selection in the TTA setting, since high-uncertainty samples are hard to learn in one step.
- Annotating one sample every three or five batches with the baseline becomes worse than the baseline annotating one sample per batch.
Limitations
- Relies on annotations from humans or large models, and delays may still appear in the TTA process.
- Experiments are limited to image classification benchmarks with ResNet and ViT backbones.
Experimental setup
Datasets: ImageNet-C ImageNet-R ImageNet-K ImageNet-A PACS
Benchmarks: ImageNet-C ImageNet-R ImageNet-K ImageNet-A PACS
Metrics: average error rate average accuracy true positive rate
Baselines: TENT CoTTA SAR ETA SimATTA CEMA HILTTA
Builds on: SimATTA TENT Entropy Margin (ETA) CEMA HILTTA
Closest papers by meaning
- Learning to Generate Gradients for Test-Time Adaptation via Test-Time Training Layers (AAAI 2025 · similarity 0.87)
- Maintaining Consistent Inter-Class Topology in Continual Test-Time Adaptation (CVPR 2025 · similarity 0.84)
- Improved Self-Training for Test-Time Adaptation (CVPR 2024 · similarity 0.84)
- Active Test-Time Adaptation: Theoretical Analyses and An Algorithm (ICLR 2024 · similarity 0.84)
- Efficient Test-Time Adaptation of Vision-Language Models (CVPR 2024 · similarity 0.83)
- Multi-Label Test-Time Adaptation with Bound Entropy Minimization (ICLR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.