Research map / Label-efficient and robust learning / Test-time adaptation
PALM: Pushing Adaptive Learning Rate Mechanisms for Continual Test-Time Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PALM, a continual test-time adaptation method that selects layers via KL-to-uniform gradient norms (no pseudo-labels) and scales per-parameter learning rates by a sensitivity-based domain-shift estimate.
Problem
Continual test-time adaptation (CTTA) of pre-trained vision classifiers under shifting corruptions suffers from error accumulation, catastrophic forgetting, and unreliable pseudo-label-based estimates of layer importance and domain shift used by adaptive learning rate methods like LAW.
Approach
Select layers to adapt by the L1 norm of gradients from backpropagating KL divergence between tempered softmax output and a uniform distribution (pseudo-label free), freeze the rest, and set per-parameter learning rates from a moving-average sensitivity |θ·∇L| based importance ratio.
Key results
- PALM achieves mean error 15.5% on CIFAR-10C, 30.1% on CIFAR-100C, and 60.1% on ImageNet-C in the CTTA setting, versus LAW's 17.2%, 31.7%, and 61.0%.
- Using pseudo-label-based layer importance instead of the KL-gradient scoring raises errors to 17.9%, 33.8%, and 68.6%.
- PALM updates on average 4.34%, 6.65%, and 17.21% of source parameters per task versus 100% for CoTTA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace pseudo-label-based layer importance with gradient-based uncertainty from KL to a uniform distribution (ablation) | PALM · CIFAR-10C, CIFAR-100C, ImageNet-C, CTTA severity 5 | mean error (%): 17.9 -> 15.5; 33.8 -> 30.1; 68.6 -> 60.1 | Helped |
| set the consistency-loss coefficient λ to 0.01 (ablation) | PALM · CIFAR-10C, CIFAR-100C, ImageNet-C, CTTA severity 5 | mean error (%): 16.8 -> 15.5; 30.4 -> 30.1; 63.9 -> 60.1 | Helped |
| vary consistency-loss coefficient λ from 0.05 to 1.0 instead of using λ=0.01 (ablation) | PALM · CIFAR-10C, CIFAR-100C, ImageNet-C, CTTA severity 5 | mean error (%): CIFAR-10C: 16.4, 17.0, 17.6, 17.9; CIFAR-100C: 30.5, 30.5, 30.6, 30.7; ImageNet-C: 60.3, 60.4, 60.4, 60.4 | Hurt |
| use LARS, Prodigy, or Adafactor instead of PALM's sensitivity-based parameter learning rates (ablation) | PALM · CIFAR-10C, CIFAR-100C, ImageNet-C, CTTA severity 5 | mean error (%): CIFAR-10C: 15.5 vs 18.9, 20.4, 73.8; CIFAR-100C: 30.1 vs 34.1, 35.5, 97.7; ImageNet-C: 60.1 vs 62.0, 68.6, 99.2 | Hurt |
| use PALM instead of the strongest reported CTTA baseline | PALM · CIFAR-10C, CIFAR-100C, ImageNet-C, CTTA severity 5 | mean error (%): 16.0 -> 15.5; 30.9 -> 30.1; 61.0 -> 60.1 | Helped |
| use PALM instead of CoTTA or SFT (cont.) | PALM · GTTA on CIFAR-10C, CIFAR-100C, ImageNet-C | mean error (%): CIFAR-10C: 10.9 -> 9.5; CIFAR-100C: 25.3 -> 26.1; ImageNet-C: 38.8 -> 38.5 | Mixed |
| use a weighted moving average of parameter sensitivity across batches (ablation) | PALM · CIFAR-C and ImageNet-C, CTTA | mean error (%) | Helped |
| increase temperature T on ImageNet-C (ablation) | PALM · ImageNet-C, CTTA | mean error (%) | Helped |
| increase temperature T to 100 or above on CIFAR-C (ablation) | PALM · CIFAR-10C and CIFAR-100C, CTTA | mean error (%) | Hurt |
| adapt only a subset of parameters selected using gradient-based uncertainty and freeze the rest | PALM · CIFAR-10C, CIFAR-100C, ImageNet-C | adapted source parameters per task (%): 4.34, 6.65, and 17.21 vs 100 | Helped |
Surprising findings
- Tent Cont. error on CIFAR-100C rises to 60.9% in the table's per-task results, showing error accumulation from entropy minimization
- Continually fine-tuning only the first convolutional block (SFT cont.) beats most CTTA baselines on CIFAR datasets
Limitations
- Adaptation time per batch is higher than LAW and SFT due to the two-stage design
- Hyperparameters (alpha, temperature T, eta, base LR) are set per dataset, with T and alpha tuned per dataset
- Accuracy declines after many tasks on ImageNet-C
Experimental setup
Datasets: CIFAR-10C CIFAR-100C ImageNet-C
Benchmarks: CIFAR-10C CIFAR-100C ImageNet-C
Metrics: mean classification error (%) percentage of adapted parameters adaptation time per batch
Baselines: BN-1 Tent Cont. CoTTA EcoTTA AdaContrast DSS LAW SFT (cont.) LARS Prodigy Adafactor
Builds on: LAW surgical fine-tuning TENT CoTTA EcoTTA
Closest papers by meaning
- Efficient Test-Time Adaptation of Vision-Language Models (CVPR 2024 · similarity 0.89)
- ViDA: Homeostatic Visual Domain Adapter for Continual Test Time Adaptation (ICLR 2024 · similarity 0.88)
- A Layer Selection Approach to Test Time Adaptation (AAAI 2025 · similarity 0.88)
- ReservoirTTA: Prolonged Test-time Adaptation for Evolving and Recurring Domains (NeurIPS 2025 · similarity 0.88)
- Towards Robust Multimodal Open-set Test-time Adaptation via Adaptive Entropy-aware Optimization (ICLR 2025 · similarity 0.87)
- A Versatile Framework for Continual Test-Time Domain Adaptation: Balancing Discriminability and Generalizability (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.