Research map / Label-efficient and robust learning / Test-time adaptation
PTTA: Purifying Malicious Samples for Test-Time Model Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PTTA, a plug-and-play test-time adaptation method that purifies malicious (filtered-out) test samples by Mixup with retrieved benign samples that have opposite saliency (logit-gradient) directions, instead of discarding them.
Problem
Test-time adaptation methods discard high-entropy or otherwise 'malicious' test samples via sample selection thresholds, wasting test data and making performance highly sensitive to thresholds and harming stability when selection is imperfect.
Approach
Use a logit-level saliency indicator (gradient of entropy w.r.t. logits) to find, for each malicious sample, the benign sample with the most opposite saliency direction (from an OOD memory bank or in-batch), then Mixup the two and add a purification loss on the mixed samples to existing TTA objectives.
Key results
- On lifelong ImageNet-C TTA, ETA+PTTA reduces the round-1 to round-10 accuracy drop from -37.1% to -1.2%, and EATA+PTTA stays flat (+0.3%).
- Tent+PTTA improves continual ImageNet-C ResNet50 accuracy from 9.9% to 36.9%, and DeYO+PTTA improves single-task ResNet50 accuracy by about 1.3 points (48.6 to 49.9).
- PTTA raises CLIP-ViT-B/16 zero-shot-based TPT averages by about 0.3-0.5 points on ImageNet variants and improves CarlaTTA segmentation mIoU by up to 1.8.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add PTTA to ETA | ETA · ImageNet-C continual, ResNet50 | top-1 accuracy: 43.7±0.06 -> 48.5±0.05 | Helped |
| add PTTA to ETA | ETA · ImageNet-C lifelong | top-1 accuracy: 39.0±15.2 -> 64.8±0.00 | Helped |
| add PTTA to DeYO | DeYO · ImageNet-C continual, ViT-B/16 | top-1 accuracy: 62.2±0.35 -> 65.8±0.01 | Helped |
| add PTTA to DeYO | DeYO · ImageNet-1K/-A/-V2/-R/-S average | top-1 accuracy: 58.7±0.13 -> 59.7±0.57 | Helped |
| add PTTA to EATA | EATA · ImageNet-C lifelong | top-1 accuracy: 61.6±0.41 -> 65.0±0.40 | Helped |
| add PTTA to MEMO-EPI. | MEMO-EPI. · CarlaTTA day2night | mIoU: 61.1±0.01 -> 61.4±0.01 | Helped |
| use the logit-saliency indicator instead of the base algorithm (ablation) | PTTA with ETA/DeYO/CPL · ImageNet-C single | top-1 accuracy: 46.5 -> 48.0 | Helped |
| use the logit-saliency indicator instead of the base algorithm (ablation) | PTTA with ETA/DeYO/CPL · ImageNet-C continual | top-1 accuracy: 60.0 -> 63.5 | Helped |
| replace Mixup purification with FGSM purification (ablation) | PTTA with ETA/DeYO/CPL · ImageNet-C single | top-1 accuracy: 46.5 -> 45.0 | Hurt |
| replace Mixup purification with SOAP purification (ablation) | PTTA with ETA/DeYO/CPL · ImageNet-C single | top-1 accuracy: 46.5 -> 43.9 | Hurt |
| retrieve benign samples from OOD memory instead of ID data (ablation) | PTTA with ETA/DeYO/CPL · ImageNet-C single | top-1 accuracy: 46.5 -> 48.0 | Helped |
| retrieve benign samples within the mini-batch instead of ID data (ablation) | PTTA with ETA/DeYO/CPL · ImageNet-C single | top-1 accuracy: 46.5 -> 47.6 | Helped |
Surprising findings
- Adversarial purification methods (FGSM, SOAP) fail to improve and can even impair TTA performance, whereas Mixup-based purification with retrieved benign samples helps.
- Tent+PTTA performs comparably with or without the sample selection criterion, suggesting selection can be dropped when the saliency indicator retrieves well.
Limitations
- Sensitive to the mixing coefficient alpha, which must be set differently for selection-based and selection-free methods
- Memory bank size matters; very large banks are unsuitable for dynamically changing environments
Experimental setup
Datasets: ImageNet-C CIFAR100-C ImageNet-1K ImageNet-A ImageNet-V2 ImageNet-R ImageNet-Sketch CarlaTTA
Benchmarks: ImageNet-C CIFAR100-C ImageNet-1K ImageNet-A ImageNet-V2 ImageNet-R ImageNet-Sketch CarlaTTA
Metrics: top-1 accuracy mIoU error rate
Baselines: Tent TPT ETA DeYO CPL SAR MEMO SoTTA CoTTA EATA FGSM purification SOAP purification
Builds on: Tent ETA EATA DeYO CPL CoTTA Mixup
Closest papers by meaning
- On the Adversarial Risk of Test Time Adaptation: An Investigation into Realistic Test-Time Data Poisoning (ICLR 2025 · similarity 0.89)
- L-TTA: Lightweight Test-Time Adaptation Using a Versatile Stem Layer (NeurIPS 2024 · similarity 0.87)
- AETTA: Label-Free Accuracy Estimation for Test-Time Adaptation (CVPR 2024 · similarity 0.86)
- MedBN: Robust Test-Time Adaptation against Malicious Test Samples (CVPR 2024 · similarity 0.86)
- A Layer Selection Approach to Test Time Adaptation (AAAI 2025 · similarity 0.86)
- Persistent Test-time Adaptation in Recurring Testing Scenarios (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.