Research map / Continual learning / Domain and prompt-based
Ranked Entropy Minimization for Continual Test-Time Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Ranked Entropy Minimization (REM), a single-model continual test-time adaptation method that uses ViT attention-based progressive masking with a masked consistency loss and an entropy ranking loss to avoid entropy-minimization model collapse.
Problem
Entropy minimization for continual test-time adaptation suffers from model collapse (predicting a single class via trivial solution) and forgetting, while consistency-regularization methods are stable but computationally expensive.
Approach
Build a chain of progressively masked images using ViT CLS-attention to mask foreground objects, then enforce cross-entropy consistency from less-masked to more-masked predictions (MCL) and a margin ranking loss keeping entropy ordered by masking ratio (ERL), updating only normalization layers of a single model.
Key results
- On ImageNet-to-ImageNetC, REM reaches 39.2% mean error versus 42.5% for Continual-MAE and 55.8% for the source model (16.6% gain).
- On CIFAR100-to-CIFAR100C, REM achieves 23.4% mean error, a 12.0 point gain over the source model, though CIFAR10-to-CIFAR10C results are best at 9.4% error.
- REM uses 0.03M trainable parameters and about 17m21s total time on ImageNetC versus 59m56s for Continual-MAE, with 3 forward passes and a single test model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use REM instead of Continual-MAE | ViT-B/16 · ImageNet-to-ImageNetC | classification error rate (%): 42.5 -> 39.2 | Helped |
| Use REM instead of Continual-MAE | ViT-B/16 · CIFAR10-to-CIFAR10C | classification error rate (%): 12.6 -> 9.4 | Helped |
| Use REM instead of SAR | ViT-B/16 · CIFAR100-to-CIFAR100C | classification error rate (%): 26.2 -> 23.4 | Helped |
| Remove masked consistency loss (MCL) (ablation) | REM · CIFARC | classification error rate | Hurt |
| Remove entropy ranking loss (ERL; called REL in the ablation description) (ablation) | REM · CIFARC | classification error rate | Hurt |
| Replace foreground masking with background or random masking (ablation) | REM · all test datasets | classification error rate | Hurt |
| Use three mask chains (N=3) rather than the adopted mask-ratio combination (ablation) | REM · ImageNet-to-ImageNetC | classification error rate | Helped |
Surprising findings
- Tent's entropy minimization on ImageNetC collapses on several domains, with error rising for some corruptions, motivating the ranking constraint.
- The best masking configuration (N=3) is not used by default; N=2 with masks {0, 5%, 10%} is chosen for computational balance.
Limitations
- No rigorous theoretical proof that masking foreground lowers accuracy and raises entropy; relies on empirical evidence.
- Adaptation speed trade-off: high learning rates help seen domains while lower rates help unseen domains; episodic time-constrained TTA performs worse than EATA.
- Failure cases where easier domains yield small masked-vs-original discrepancy and slower adaptation (e.g., Brightness, JPEG).
Experimental setup
Datasets: ImageNet ImageNetC CIFAR10 CIFAR10C CIFAR100 CIFAR100C ImageNet-R ImageNet-V2 ImageNet-Sketch ImageNet-3DCC VisDA-C Office-Home
Benchmarks: ImageNet-to-ImageNetC CIFAR10-to-CIFAR10C CIFAR100-to-CIFAR100C ImageNet-R/V2/Sketch ImageNetC online imbalanced label shift ImageNetC batch size 1 ImageNetC mixed shifts ImageNet-3DCC CLIP TTA benchmarks (CIFAR, VisDA-C, Office-Home, PACS, VLCS)
Metrics: classification error rate classification accuracy mean error expected calibration error training time
Baselines: Source Pseudo-label Tent CoTTA VDP SAR PETAL ViDA Continual-MAE EATA DeYO MEMO
Builds on: Tent Continual-MAE CoTTA SAR EATA
Closest papers by meaning
- Entropy Minimization without Model Collapse: Mitigating Prediction Bias in Medical Imaging (NeurIPS 2026 · similarity 0.87)
- Protected Test-Time Adaptation via Online Entropy Matching: A Betting Approach (NeurIPS 2024 · similarity 0.87)
- COME: Test-time Adaption by Conservatively Minimizing Entropy (ICLR 2025 · similarity 0.87)
- Continual-MAE: Adaptive Distribution Masked Autoencoders for Continual Test-Time Adaptation (CVPR 2024 · similarity 0.87)
- Unified Entropy Optimization for Open-Set Test-Time Adaptation (CVPR 2024 · similarity 0.86)
- Beyond Entropy: Region Confidence Proxy for Wild Test-Time Adaptation (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.