Research map / Label-efficient and robust learning / Test-time adaptation

Entropy Minimization without Model Collapse: Mitigating Prediction Bias in Medical Imaging

NeurIPS 2026 · Label-efficient and robust learning › Test-time adaptation · approach: Extrapolation and single-sample adaptation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DSBR, a test-time entropy-minimization objective that reweights each sample's entropy loss by the inverse of its estimated predicted-class frequency (EMA-smoothed), to counter prediction bias that drives model collapse under distribution shift, evaluated on four medical imaging datasets and ImageNet-C.

Problem

Entropy minimization for test-time adaptation often collapses to a single class under distribution shift; the mechanism behind this failure is poorly understood, especially for subtle medical imaging shifts.

Approach

Distribution shift merges feature clusters while the classifier boundary stays fixed, creating prediction bias that entropy minimization amplifies; DSBR importance-reweights the entropy loss by 1/(K * P_hat(predicted class)) so each predicted class contributes equally, using an EMA estimate of class frequencies computed at test time.

Key results

• DSBR achieves the highest average improvement on medical datasets: +4.2% for ResNet50-GN and +2.6% for ViT-B/16, versus +1.7% and +2.3% for the next-best baseline (ROID).
• On ImageNet-C severity 5 default setting, DSBR (alpha=0.99) reaches 49.7% for ResNet50-GN versus 31.4% non-adapted and 32.2% Tent; DSBR gives +13.9% average improvement over ResNet50-GN.
• Tent collapses to near chance (~50%) on a substantial fraction of WILDSCamelyon hospital pairs, while DSBR shows no collapses across configurations.

Surprising findings

• Initial gradient norm has no effect on entropy minimization performance, contrary to prior attribution of collapse to high-gradient samples.
• Re-implementations of Tent on ImageNet-C with pip packages obtain higher numbers than those reported in prior work, suggesting reproducibility discrepancies in baselines.

Limitations

• DSBR relies on accurate estimation of predicted-class distribution, which degrades under imbalanced label shifts.
• Performance degrades with very small batch sizes relative to number of classes unless EMA is used.
• On GliomaMRI, DSBR is not the top performer (ROID is best on ResNet and ViT results are close to baseline on ViT).

Experimental setup

Datasets: WILDSCamelyon Histopantum MammoBench GliomaMRI ImageNet-C

Benchmarks: WILDSCamelyon Histopantum MammoBench GliomaMRI ImageNet-C

Metrics: balanced accuracy ROC-AUC

Baselines: Tent SAR ROID DeYO EATA COME

Builds on: Tent DeYO ROID SAR EATA

Closest papers by meaning

• Ranked Entropy Minimization for Continual Test-Time Adaptation (ICML 2025 · similarity 0.87)
• The Entropy Enigma: Success and Failure of Entropy Minimization (ICML 2024 · similarity 0.87)
• Entropy is not Enough for Test-Time Adaptation: From the Perspective of Disentangled Factors (ICLR 2024 · similarity 0.87)
• Unified Entropy Optimization for Open-Set Test-Time Adaptation (CVPR 2024 · similarity 0.86)
• Decoupled Entropy Minimization (NeurIPS 2025 · similarity 0.85)
• Test-Time Adaptation on Noisy Data via Model-Pruning-Based Filtering and Flatness-Aware Entropy Minimization (AAAI 2025 · similarity 0.85)