Research map / Label-efficient and robust learning / Test-time adaptation
Learning to Generate Gradients for Test-Time Adaptation via Test-Time Training Layers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MGTTA, which learns a meta gradient generator (MGG) with a gradient memory layer (a test-time-training-style layer) to refine noisy TTA gradients on ViT-Base, achieving SOTA on ImageNet-C/R/Sketch/A with fewer updates.
Problem
Test-time adaptation with unsupervised objectives such as entropy minimization yields noisy, unreliable gradients that slow convergence and destabilize online adaptation.
Approach
Replace the hand-designed SGD update in TTA with a learned optimizer (MGG) that uses a gradient memory layer, trained via a self-supervised reconstruction loss, to compress historical gradients into parameters and output refined gradients for the current step.
Key results
- On ImageNet-C (severity 5, ViT-Base), MGTTA reaches 71.3% average accuracy vs. 68.2% for DeYO and 62.7% for SAR.
- With only 35 of 782 batches adapted, MGTTA achieves 69.3% on ImageNet-C, above all baselines using full data.
- On ImageNet-R/Sketch/A, MGTTA averages 60.1% vs. 57.5% for DeYO, and adds only about 28 MB GPU memory over TENT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MGTTA instead of DeYO | ViT-Base test-time adaptation · ImageNet-C severity 5 | accuracy: 68.2% vs. 71.3% | Helped |
| use MGTTA instead of DeYO | ViT-Base test-time adaptation · ImageNet-R | accuracy: 66.1 -> 70.2 | Helped |
| use MGTTA instead of DeYO | ViT-Base test-time adaptation · ImageNet-Sketch | accuracy: 52.2 -> 53.3 | Helped |
| use MGTTA instead of DeYO | ViT-Base test-time adaptation · ImageNet-A | accuracy: 54.1 -> 56.7 | Helped |
| add MGG gradient optimization (ablation) | MGTTA · ImageNet-C severity 5 | accuracy: 70.0 -> 71.3 | Helped |
| add MGG gradient optimization (ablation) | MGTTA · ImageNet-R | accuracy: 67.2 -> 70.2 | Helped |
| add MGG gradient optimization (ablation) | MGTTA · ImageNet-Sketch | accuracy: 51.9 -> 53.3 | Helped |
| add MGG gradient optimization (ablation) | MGTTA · ImageNet-A | accuracy: 55.1 -> 56.7 | Helped |
| replace LSTM with GML (ablation) | MGTTA · ImageNet-C severity 5 | accuracy: 71.3 vs. 71.3 | No clear effect |
| replace LSTM with GML (ablation) | MGTTA · ImageNet-R | accuracy: 69.4 -> 70.2 | Helped |
| replace LSTM with GML (ablation) | MGTTA · ImageNet-Sketch | accuracy: 50.3 -> 53.3 | Helped |
| replace LSTM with GML (ablation) | MGTTA · ImageNet-A | accuracy: 56.8 -> 56.7 | Hurt |
Surprising findings
- Replacing the gradient memory layer with an LSTM gives similar ImageNet-C accuracy (71.3%) but lower ImageNet-R/Sketch performance, which the authors attribute to MGG being trained on ImageNet-C data.
Limitations
- Gains in some settings are modest (e.g., ImageNet-A is roughly tied with the LSTM variant).
- MGG is pre-trained on ImageNet-C validation data, so transfer to other shifts depends on that choice.
- Hyperparameters for EATA and DeYO were re-tuned by the authors, so baseline comparisons may not be fully equal.
Experimental setup
Datasets: ImageNet-C validation set (128 unlabeled samples) ImageNet-C test set ImageNet-R ImageNet-Sketch ImageNet-A
Benchmarks: ImageNet-C ImageNet-R ImageNet-Sketch ImageNet-A
Metrics: accuracy wall-clock runtime GPU memory
Baselines: NoAdapt LAME T3A TENT CoTTA SAR FOA EATA DeYO
Builds on: FOA test-time training layers learning to optimize Adam
Closest papers by meaning
- A Layer Selection Approach to Test Time Adaptation (AAAI 2025 · similarity 0.87)
- Effortless Active Labeling for Long-Term Test-Time Adaptation (CVPR 2025 · similarity 0.87)
- Maintaining Consistent Inter-Class Topology in Continual Test-Time Adaptation (CVPR 2025 · similarity 0.85)
- Towards Real-World Test-Time Adaptation: Tri-net Self-Training with Balanced Normalization (AAAI 2024 · similarity 0.84)
- Buffer layers for Test-Time Adaptation (NeurIPS 2025 · similarity 0.84)
- Active Test-Time Adaptation: Theoretical Analyses and An Algorithm (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.