Research map / Label-efficient and robust learning / Test-time adaptation
Test-Time Adaptation by Causal Trimming
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TACT, a backpropagation-free test-time adaptation method that uses PCA over augmented test representations to find non-causal directions, then trims them from sample representations and class prototypes, optionally combined with gradient-based TACT-adapt.
Problem
Test-time adaptation degrades under distribution shift because models rely on non-causal features, which make predictions and pseudo-labels unreliable.
Approach
Identify non-causal directions as top principal components of representations of augmented test samples (augmentations preserve causal content), then project them out of both test representations and class prototypes, with a moving average of trimmed prototypes; TACT-adapt uses TACT predictions as pseudo-labels for gradient updates.
Key results
- TACT outperforms backpropagation-free baselines on all five datasets, e.g., about 4% gain on Birdcalls, 15% on CivilComments, and 1.7% on ImageNet-R.
- TACT-adapt achieves the best results on all datasets, e.g., 83.70% on Camelyon17 and 48.81% on ImageNet-R.
- Ablation shows trimming representations alone (without prototype trimming/averaging) gives gains, and the full combination performs best.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply TACT causal trimming | ViT-B/32 · Birdcalls | macro F1: 26.16$\pm$1.33 -> 31.14$\pm$1.69 | Helped |
| Apply TACT causal trimming | ViT-B/32 · Camelyon17 | accuracy: 69.96$\pm$1.98 -> 70.17$\pm$0.05 | No clear effect |
| Apply TACT causal trimming | DistilBERT · CivilComments | worst-group accuracy: 56.43$\pm$0.00 -> 71.80$\pm$0.35 | Helped |
| Apply TACT causal trimming | ViT-B/32 · ImageNet-R | accuracy: 41.78$\pm$0.12 -> 43.59$\pm$0.02 | Helped |
| Apply TACT causal trimming | ViT-B/32 · ImageNet-V2 | accuracy: 62.93$\pm$0.02 -> 63.33$\pm$0.10 | Helped |
| Use TACT predictions to guide gradient-based updates | ViT-B/32 · Birdcalls | macro F1: 31.14$\pm$1.69 -> 31.25$\pm$3.59 | No clear effect |
| Use TACT predictions to guide gradient-based updates | ViT-B/32 · Camelyon17 | accuracy: 70.17$\pm$0.05 -> 83.70$\pm$1.10 | Helped |
| Use TACT predictions to guide gradient-based updates | DistilBERT · CivilComments | worst-group accuracy: 71.80$\pm$0.35 -> 71.98$\pm$0.19 | No clear effect |
| Use TACT predictions to guide gradient-based updates | ViT-B/32 · ImageNet-R | accuracy: 43.59$\pm$0.02 -> 48.81$\pm$0.05 | Helped |
| Use TACT predictions to guide gradient-based updates | ViT-B/32 · ImageNet-V2 | accuracy: 63.33$\pm$0.10 -> 63.44$\pm$0.07 | No clear effect |
| Add trimmed representation while retaining trimmed, averaged prototypes (ablation) | TACT · Birdcalls | macro F1: 27.36$\pm$0.23 -> 31.14$\pm$1.69 | Helped |
| Add trimmed representation while retaining trimmed, averaged prototypes (ablation) | TACT · Camelyon17 | accuracy: 64.74$\pm$0.05 -> 70.17$\pm$0.05 | Helped |
Surprising findings
- Removing the first principal component on Camelyon17 degraded performance, suggesting important causal features may lie in it.
- Tent and SAR show no change on CivilComments (55.38), and SHOT collapses to 13.93 on CivilComments worst-group accuracy.
Limitations
- Requires prior knowledge to choose augmentations that vary non-causal features without altering causal ones.
- PCA's orthogonality constraint may not capture non-causal features well; causal and non-causal features are not explicitly disentangled in the representation space.
- Performance is sensitive to augmentation choice and number of removed components m.
Experimental setup
Datasets: Birdcalls Camelyon17 CivilComments ImageNet-R ImageNet-V2 ImageNet
Benchmarks: Birdcalls Camelyon17 CivilComments ImageNet-R ImageNet-V2
Metrics: macro F1 accuracy worst-group accuracy
Baselines: No TTA T3A LAME FOA SHOT Tent SAR DeYO TAST TSD PASLE
Builds on: T3A DeYO SHOT Tent WILDS
Closest papers by meaning
- A Layer Selection Approach to Test Time Adaptation (AAAI 2025 · similarity 0.89)
- TEA: Test-time Energy Adaptation (CVPR 2024 · similarity 0.87)
- Backpropagation-Free Test-Time Adaptation via Probabilistic Gaussian Alignment (NeurIPS 2025 · similarity 0.87)
- AETTA: Label-Free Accuracy Estimation for Test-Time Adaptation (CVPR 2024 · similarity 0.87)
- SNAP: Low-Latency Test-Time Adaptation with Sparse Updates (NeurIPS 2025 · similarity 0.86)
- Unified Entropy Optimization for Open-Set Test-Time Adaptation (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.