Research map / Label-efficient and robust learning / Domain adaptation and generalization

Consistency-Guided Temperature Scaling Using Style and Content Information for Out-of-Domain Calibration

AAAI 2024 · Label-efficient and robust learning › Domain adaptation and generalization · approach: Mixture and distillation for domain shift · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes consistency-guided temperature scaling (CTS), a post-hoc calibration method that adds inter-domain style and intra-class content consistency KL losses to the temperature-scaling NLL objective, using only source-domain validation data, to improve out-of-domain calibration.

Problem

Temperature scaling works in-domain but degrades under domain shift because no validation set from the unseen target domain is available, leading to over-confident predictions on OOD samples.

Approach

Optimize a single temperature with NLL plus two KL consistency losses computed on style-swapped (AdaIN-style feature statistics from another source sample) and content-swapped (same-class sample content) logits, so the calibrated predictions become invariant to style and content shifts using only source-domain validation samples.

Key results

• On PACS, CTS reduces average ECE to 4.31% versus 9.06% for TS with val and 9.83% for vanilla (one-domain-leave-out).
• On Digits-DG, CTS achieves average ECE 6.53% vs 11.08% for TS with val; on Office-Home 4.75% vs 9.79%.
• On VLCS, CTS achieves 12.90% average ECE vs 14.00% for TS with val, the best among post-hoc methods without target data.

Measured effects reported in the paper

Surprising findings

• Class-wise consistency (minimizing confidence differences between same-class samples) alone already improves over TS with val on all datasets.
• Post-hoc calibration can hurt calibration when the target domain is Photo in PACS, given ImageNet pre-training.

Limitations

• Gap to TS with test (oracle using target-domain data) remains large on several datasets.
• Coefficients lambda1 and lambda2 must be tuned per dataset using the validation set.
• Evaluated on ResNet-18 with features taken from the first residual block only, with limited architecture coverage in main text.

Experimental setup

Datasets: PACS Office-Home Digits-DG VLCS

Benchmarks: PACS VLCS Office-Home Digits-DG

Metrics: expected calibration error (ECE) accuracy

Baselines: Vanilla TS with val TS with test PerturbTS MDTS CCDG-NN CCDG-Reg CCDG-Ens

Builds on: temperature scaling adaptive instance normalization CCDG MDTS PerturbTS

Closest papers by meaning

• T-CIL: Temperature Scaling using Adversarial Perturbation for Calibration in Class-Incremental Learning (CVPR 2025 · similarity 0.87)
• On the Limitations of Temperature Scaling for Distributions with Overlaps (ICLR 2024 · similarity 0.87)
• A Benchmark Study on Calibration (ICLR 2024 · similarity 0.86)
• On Temperature Scaling and Conformal Prediction of Deep Classifiers (ICML 2025 · similarity 0.86)
• GETS: Ensemble Temperature Scaling for Calibration in Graph Neural Networks (ICLR 2025 · similarity 0.86)
• HoTS: Homophily-Aware Temperature Scaling for Graph Neural Network Calibration (NeurIPS 2026 · similarity 0.85)