Research map / Architectures and efficiency / Pruning and efficient tuning
Asymmetric Duos: Sidekicks Improve Uncertainty
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Asymmetric Duos, pairing a large fine-tuned base model with a much smaller sidekick via learned two-temperature logit averaging, to improve accuracy, uncertainty quantification, and selective classification at ~10-20% extra FLOPs.
Problem
Deep ensembles are too expensive for large pre-trained models that are fine-tuned, and fine-tuned members give limited ensemble gains; a cheap way to improve uncertainty quantification and downstream decisions is needed.
Approach
Combine a large base model with a much smaller, cheaper sidekick model via a weighted logit average whose two temperatures are tuned on a validation set by NLL; the weighting lets the sidekick help even when much less accurate, and it can reduce to the base model if the sidekick is not useful.
Key results
- Asymmetric Duos improve accuracy, AUROC, AURC and SAC@98 over the single base model with roughly 10-20% extra FLOPs, with gains up to about 10-15%.
- On ImageNet, a RN50+ENB2 duo (27% FLOPs balance) reaches 81.87% accuracy versus 81.97% for a 5-member ResNet-50 deep ensemble, which uses ~4x the FLOPs.
- Unweighted duos significantly underperform the base model when the FLOPs balance is near zero, so the learned temperature weighting is necessary.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a smaller sidekick and use learned weighted logit averaging | ResNet-50 · ImageNet | accuracy: 79.82$\pm$0.51 -> 81.87$\pm$0.19 | Helped |
| Add a smaller sidekick and use learned weighted logit averaging | ResNet-50 · ImageNet | AUROC: 86.62$\pm$0.49 -> 87.15$\pm$0.26 | Helped |
| Add a smaller sidekick and use learned weighted logit averaging | ResNet-50 · ImageNet | AURC: 5.72$\pm$0.38 -> 4.80$\pm$0.17 | Helped |
| Add a sidekick with learned weighted averaging | ResNet-50 · Caltech 256 | accuracy: 87.72 -> 89.75±1.17 | Helped |
| Add a sidekick with learned weighted averaging | ResNet-50 · Caltech 256 | AURC: 2.01 -> 1.52±0.18 | Helped |
| Add a sidekick with learned weighted averaging | Swin-V2-S · Caltech 256 | accuracy: 91.7 -> 93.0±0.3 | Helped |
| Add a sidekick with learned weighted averaging | Swin-V2-S · Caltech 256 | AUROC: 93.6 -> 93.4±0.2 | No clear effect |
| Use equal weights instead of learned asymmetric weights (ablation) | Asymmetric Duos · ImageNet V2, Caltech 256, iWildCam | accuracy | Hurt |
| Use the Duo uncertainty estimate but retain the base model's class prediction (ablation) | Asymmetric Duos · ImageNet V2, Caltech 256, iWildCam | AUROC | Helped |
| Use the Duo uncertainty estimate but retain the base model's class prediction (ablation) | Asymmetric Duos · ImageNet V2, Caltech 256, iWildCam | AURC | Helped |
| Use the Duo uncertainty estimate but retain the base model's class prediction (ablation) | Asymmetric Duos · ImageNet V2, Caltech 256, iWildCam | SAC@98 | Helped |
Surprising findings
- A sidekick that is 10x smaller and about 80% as accurate still almost never harms the base model when temperatures are learned.
- Duos improve AUROC and AURC even at near-zero FLOPs balance, and UQ-only duos (base-model predictions with duo uncertainty) recover most of the gain, so the gain is not just from accuracy.
Limitations
- Evaluated only on image classification across five test sets; segmentation, regression and NLP are unexplored.
- Aggregation is limited to two learned temperatures; more adaptive or learned weighting schemes are not explored.
- Relies on a held-out validation set for temperature tuning, and the sidekick and base are both fine-tuned on the same data.
Experimental setup
Datasets: ImageNet Caltech 256 iWildCam ImageNet V2 iWildCam-OOD
Benchmarks: ImageNet ImageNet V2 Caltech 256 iWildCam iWildCam-OOD
Metrics: accuracy macro F1 Brier score negative log likelihood ECE AUROC AURC SAC@98
Baselines: single base model (temperature scaled) unweighted duo deep ensembles (m=2, m=5, m=8) IVON Laplace Approximation model soup shallow ensembles
Builds on: deep ensembles temperature scaling model soups LP-FT shallow ensembles
Closest papers by meaning
- Deep Deterministic Uncertainty: A New Simple Baseline (CVPR 2023 · similarity 0.86)
- What Can we Learn From The Selective Prediction And Uncertainty Estimation Performance Of 523 Imagenet Classifiers? (ICLR 2023 · similarity 0.86)
- SURE: SUrvey REcipes for building reliable and robust deep networks (CVPR 2024 · similarity 0.85)
- Quantifying Uncertainty in Error Consistency: Towards Reliable Behavioral Comparison of Classifiers (NeurIPS 2025 · similarity 0.84)
- Directional Confusions Reveal Divergent Inductive Biases Through Rate-Distortion Geometry in Human and Machine Vision (NeurIPS 2026 · similarity 0.84)
- Exploring and Exploiting the Asymmetric Valley of Deep Neural Networks (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.