Research map / Architectures and efficiency / Pruning and efficient tuning

SURGEON: Memory-Adaptive Fully Test-Time Adaptation via Dynamic Activation Sparsity

CVPR 2025 · Architectures and efficiency › Pruning and efficient tuning · approach: Joint pruning and quantization · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SURGEON, a fully test-time adaptation method that prunes cached activations at layer-specific dynamic ratios (based on gradient importance and activation memory) to cut adaptation memory while keeping accuracy gains, without architecture changes or retraining.

Problem

Backpropagation-based test-time adaptation is memory-intensive, mainly due to cached activations, which hinders deployment on resource-constrained devices; existing memory-efficient TTA methods either need training-procedure changes or rely on BN layers.

Approach

Layer-wise dynamic activation pruning: each layer's pruning ratio is set from a combined importance score of weight-gradient magnitude (accuracy contribution) and log activation size (memory cost), recomputed per test batch, with sparse activations cached in index+value form.

Key results

• On CIFAR10-C with WideResNet-28, SURGEON reaches 18.1% mean online error with 325 MB cache versus 18.9% and 3697 MB for Full Tuning (about 91% cache reduction).
• On CIFAR100-C with ResNeXt-29, SURGEON gets 33.1% mean error at 820 MB, vs. 34.3% at 5403 MB for Full Tuning and 32.5% for CoTTA at 5403 MB.
• On Jetson Xavier NX (ResNeXt-29, CIFAR100-C), SURGEON achieves 37.8% error with 22.4 MB cache, compared with CoTTA at 37.5% error and 216 MB cache.

Measured effects reported in the paper

Surprising findings

• On DeeplabV3+ (ACDC), all chosen TTA baselines fail to beat the source model's accuracy.

Limitations

• Requires an additional forward-backward pass per batch to compute importance metrics, adding compute overhead (GFLOPs 28.3 vs 16.4 for MECTA on Jetson).
• Not uniformly best: on ImageNet-C, EcoTTA is slightly more accurate (54.2 vs 55.5 error) and on CIFAR10-C some baselines match or beat it on accuracy.
• Method evaluated only on fixed test-time corruption sequences and specific architectures; no evaluation beyond these benchmarks.

Experimental setup

Datasets: CIFAR10-C CIFAR100-C ImageNet-C ACDC Cityscapes

Benchmarks: CIFAR10-C CIFAR100-C ImageNet-C ACDC

Metrics: online error (%) mean online error (%) cache size (MB) GFLOPs latency (ms)

Baselines: Source BN-stat Full Tuning TENT CoTTA EcoTTA MECTA EATA Gradient checkpointing

Builds on: TENT MECTA EcoTTA CoTTA gradient checkpointing

Closest papers by meaning

• A Layer Selection Approach to Test Time Adaptation (AAAI 2025 · similarity 0.89)
• L-TTA: Lightweight Test-Time Adaptation Using a Versatile Stem Layer (NeurIPS 2024 · similarity 0.88)
• SNAP: Low-Latency Test-Time Adaptation with Sparse Updates (NeurIPS 2025 · similarity 0.87)
• Buffer layers for Test-Time Adaptation (NeurIPS 2025 · similarity 0.87)
• Test-Time Adaptation with Binary Feedback (ICML 2025 · similarity 0.87)
• Backpropagation-Free Test-Time Adaptation via Probabilistic Gaussian Alignment (NeurIPS 2025 · similarity 0.87)