Research map / Label-efficient and robust learning / Test-time adaptation
Entropy is not Enough for Test-Time Adaptation: From the Perspective of Disentangled Factors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows entropy is an unreliable TTA confidence metric under spurious correlation shifts and proposes DeYO, which filters and weights samples by entropy plus a shape-destruction-based Pseudo-Label Probability Difference (PLPD) score.
Problem
Entropy-based sample selection in test-time adaptation (TTA) fails to identify harmful samples under distribution shifts such as spurious correlations, causing error accumulation.
Approach
Introduce PLPD, the drop in pseudo-label probability after a patch-shuffle that destroys object shape, as a confidence metric capturing reliance on shape (CPR) factors; combine it with entropy for both sample selection and weighting in a fully test-time adaptation method.
Key results
- On ColoredMNIST, DeYO reaches 67.39% worst-group accuracy versus 20.05% for the ResNet-18-BN source model; it is the only method above random guessing (50%).
- On ImageNet-C severity 5 mild scenario, DeYO averages 48.6% accuracy with ResNet-50-BN, versus 47.8% for EATA and 43.1% for Tent.
- Under batch size 1 on ImageNet-C severity 5 with ViT-Base-LN, DeYO improves over SAR by about 7.8 points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add PLPD-based selection and weighting to entropy-based adaptation (ablation) | DeYO · ImageNet-C severity 5, mild | accuracy (%): 47.35 -> 48.60 | Helped |
| replace entropy selection and weighting with PLPD selection and weighting (ablation) | DeYO · ImageNet-C severity 5, mild | accuracy (%): 47.35 -> 46.37 | Hurt |
| use entropy for selection and PLPD for weighting (ablation) | DeYO · ImageNet-C severity 5, mild | accuracy (%): 47.35 -> 46.06 | Hurt |
| use PLPD for selection and entropy for weighting (ablation) | DeYO · ImageNet-C severity 5, mild | accuracy (%): 46.37 -> 47.73 | Helped |
| use entropy and PLPD for selection, with entropy weighting only (ablation) | DeYO · ImageNet-C severity 5, mild | accuracy (%): 44.68 -> 47.95 | Helped |
| use entropy and PLPD for selection, with PLPD weighting only (ablation) | DeYO · ImageNet-C severity 5, mild | accuracy (%): 44.68 -> 46.38 | Helped |
| use PLPD-only selection and add entropy weighting (ablation) | DeYO · ImageNet-C severity 5, mild | accuracy (%): 44.68 -> 47.73 | Helped |
| use PLPD-only selection and add PLPD weighting (ablation) | DeYO · ImageNet-C severity 5, mild | accuracy (%): 44.68 -> 46.37 | Helped |
| use entropy-only selection and add PLPD weighting (ablation) | DeYO · ImageNet-C severity 5, mild | accuracy (%): 44.28 -> 46.06 | Helped |
| use entropy-only selection and add entropy weighting (ablation) | DeYO · ImageNet-C severity 5, mild | accuracy (%): 44.28 -> 47.35 | Helped |
| use patch-shuffling instead of pixel-shuffling or center occlusion (ablation) | DeYO · ImageNet-C severity 5 | accuracy | Helped |
| use 4×4 patches for patch-shuffling (ablation) | DeYO · ImageNet-C severity 5 | accuracy | Helped |
Surprising findings
- Entropy-based adaptation can underperform the pre-adaptation model on Waterbirds, and the lowest-entropy samples in the worst group can have lower accuracy than higher-entropy ones.
- On Waterbirds, using PLPD alone for selection outperforms combining it with entropy, which the authors attribute to entropy's unreliability under severe shifts.
Limitations
- Relies on a patch-shuffle transformation as a proxy for shape information, which may not capture all causal factors.
- Hyperparameters such as the PLPD threshold and the learning rate needed tuning per scenario (e.g., 5x learning rate on Waterbirds).
Experimental setup
Datasets: ImageNet-C ColoredMNIST Waterbirds ImageNet-R VisDA-2021
Benchmarks: ImageNet-C ColoredMNIST Waterbirds ImageNet-R VisDA-2021
Metrics: accuracy worst-group accuracy AURC
Baselines: MEMO Tent EATA SAR SENTRY
Builds on: Tent EATA SAR MEMO
Closest papers by meaning
- Unified Entropy Optimization for Open-Set Test-Time Adaptation (CVPR 2024 · similarity 0.88)
- A Layer Selection Approach to Test Time Adaptation (AAAI 2025 · similarity 0.88)
- L-TTA: Lightweight Test-Time Adaptation Using a Versatile Stem Layer (NeurIPS 2024 · similarity 0.87)
- Entropy Minimization without Model Collapse: Mitigating Prediction Bias in Medical Imaging (NeurIPS 2026 · similarity 0.87)
- COME: Test-time Adaption by Conservatively Minimizing Entropy (ICLR 2025 · similarity 0.86)
- Test-Time Adaptation on Noisy Data via Model-Pruning-Based Filtering and Flatness-Aware Entropy Minimization (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.