Research map / Architectures and efficiency / RNNs and normalization
Beyond Greedy Exits: Improved Early Exit Decisions for Risk Control and Reliability
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UAT, a UCB multi-armed-bandit method that adapts early-exit thresholds online and unsupervised using a learned reliability score on exit confidence, with a risk bound and reported 1.70-2.10x speedup at under 2% performance drop.
Problem
Early-exit DNNs use static, validation-tuned confidence thresholds that are overconfident on wrong classes and fail under distribution shift, leading to risky exits and unreliable predictions.
Approach
Augment exit confidence with a learned reliability function g (trained jointly with exit classifiers via a modified loss) and select the exit threshold online with a UCB bandit whose reward is confidence times reliability minus a per-layer latency penalty.
Key results
- On BERT-large GLUE/SQuAD, the method reaches 1.85x speedup with accuracy at or above the full model on several tasks (e.g., SST-2 95.8 vs 95.2).
- On T5-large language tasks, it reaches 2.10x average speedup, the highest among compared methods.
- On BLIP-2 COCO captioning under Gaussian noise (sigma=1.0), B4 drops to 35.0 vs 36.4 for the full model while speedup is 1.81x.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use UAT instead of the strongest reported baseline | BERT-large early-exit inference · SST-2 | accuracy: 95.5 -> 95.8 | Helped |
| use UAT instead of the strongest reported baseline | T5-large early-exit inference · SamSum | ROUGE-L: 46.79 -> 48.09 | Helped |
| use UAT instead of the strongest reported baseline | BLIP-2-ViT-FlanT5-xl early-exit inference · COCO Karpathy test | BLEU-4: 42.5 -> 43.3 | Helped |
| add reliability calibration to confidence in the reward (ablation) | UAT reward on COCO test split · COCO test split | BLEU-4: 40.72 -> 43.32 | Helped |
| add confidence to reliability-only reward (ablation) | UAT reward on SamSum · SamSum | ROUGE-L: 48.16 -> 48.34 | Helped |
| add layer penalty to the confidence × reliability reward (ablation) | UAT reward on SamSum · SamSum | ROUGE-L: 48.34 -> 48.09 | Hurt |
| add layer penalty to the confidence × reliability reward (ablation) | UAT reward on SamSum · SamSum | speedup: 1.35× -> 1.98× | Helped |
| add reliability and coverage components to cross-entropy loss (ablation) | T5-large final-layer model · SamSum | ROUGE-L: 48.84 -> 48.82 | No clear effect |
| add reliability and coverage components to cross-entropy loss (ablation) | BLIP-2 final-layer model · COCO | BLEU-4: 43.9 -> 44.0 | No clear effect |
| add reliability and coverage components to cross-entropy loss (ablation) | BERT-large final-layer model · QNLI | accuracy: 90.6 -> 90.5 | No clear effect |
| increase epsilon from 0.01 to 0.1 (ablation) | UAT on SamSum · SamSum | speedup: 2.08× -> 2.52× | Helped |
| increase epsilon from 0.01 to 0.1 (ablation) | UAT on SamSum · SamSum | risk: 01.5 -> 03.1 | Hurt |
Surprising findings
- The method sometimes exceeds full-model accuracy on GLUE tasks, attributed to overthinking in the full model.
- Maximizing raw confidence alone or reliability alone gives worse efficiency-accuracy trade-offs than their product with the latency penalty (ablation).
Limitations
- Performance depends on the quality of the learned reliability function g.
- Uses a single global threshold across all exits rather than per-layer thresholds.
- Risk guarantee relies on assumptions (reliability approximates correctness probability, regret bound with a log T term) that are loosely argued.
Experimental setup
Datasets: GLUE SQuAD 2.0 SAMSum BIGPATENT Multi-News CNN/DailyMail IWSLT De-En COCO NoCaps VQAv2 VisDial
Benchmarks: GLUE SQuAD 2.0 SAMSum BIGPATENT Multi-News CNN/DailyMail IWSLT De-En COCO NoCaps VQAv2 VisDial
Metrics: accuracy F1 ROUGE-L BLEU-4 CIDEr SPICE METEOR VQA accuracy MRR speedup
Baselines: PABEE ZTW MuE JEI-DNN BEEM CALM MIE LAP EERC EERS
Builds on: BranchyNet PABEE EERC CALM MIE-LAP
Closest papers by meaning
- Fast yet Safe: Early-Exiting with Risk Control (NeurIPS 2024 · similarity 0.90)
- Jointly-Learned Exit and Inference for a Dynamic Neural Network (ICLR 2024 · similarity 0.86)
- BEEM: Boosting Performance of Early Exit DNNs using Multi-Exit Classifiers as Experts (ICLR 2025 · similarity 0.85)
- How to Train Your Multi-Exit Model? Analyzing the Impact of Training Strategies (ICML 2025 · similarity 0.84)
- Prudent-Banker: No Extra Fees for Baseline Safety in Adversarial Bandits With and Without Delays (NeurIPS 2026 · similarity 0.83)
- Rethinking the Adversarial Robustness of Multi-Exit Neural Networks in an Attack-Defense Game (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.