Research map / Architectures and efficiency / RNNs and normalization
Fast yet Safe: Early-Exiting with Risk Control
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adapts risk control frameworks (CRC, UCB with a relaxed WSR bound) to early-exit neural networks to choose a post-hoc exit threshold that keeps performance-gap and consistency risks (on predictions and Brier-scored distributions) under user-specified levels.
Problem
Early-exit neural networks trade inference speed against prediction quality, and it is unclear how to pick exit thresholds so that early exits do not severely degrade performance.
Approach
Formalises EENNs as threshold-based risk-controlling predictors and adapts conformal risk control (in expectation) and a high-probability UCB with a relaxed Waudby-Smith-Ramdas bound (losses in [-B,B]) to early exiting, using relative early-exit losses and Brier-based risks for predictive distributions.
Key results
- On ImageNet with n=100, controlling the prediction gap risk at epsilon=0.05 yields about 61% fewer layers for CRC and about 46% for UCB.
- On CNN/DM and SQuAD, UCB yields larger efficiency gains than LTT (used by CALM) under the same high-probability guarantee, especially at n=100.
- Test risk remains controlled across image classification, segmentation, language modeling, diffusion, and speculative decoding settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase the tolerated risk level ε | UCB early-exiting on ADP-C · Cityscapes | relative improvement over last-layer mean exit layer (%) | Helped |
| Control Brier-based predictive-distribution risks instead of prediction risks | UCB early-exiting on ADP-C · Cityscapes | relative improvement over last-layer mean exit layer (%) | Hurt |
| Use UCB instead of LTT for high-probability risk control | Early-exit T5 · CNN/DM and SQuAD | computational savings | Helped |
| Use CRC instead of UCB risk control | ImageNet early-exit models · ImageNet; prediction gap risk, ε=0.05 | layers evaluated reduction: ~ 61% for CRC; ~ 46% for UCB | Helped |
Surprising findings
- Unrestricted (negative) relative losses allow rewarding overthinking early exits, which yields less conservative thresholds than zero-bounded losses used by LTT.
- Recommending UCB over LTT even though LTT is designed to avoid the monotonicity assumption, since monotonicity is typically empirically satisfied.
Limitations
- Relies on a single shared exit threshold across layers
- Risk control holds only marginally over observations, not exit-conditionally
- Assumes i.i.d. calibration and test data and marginal monotonicity of the EENN
Experimental setup
Datasets: ImageNet Cityscapes GTA5 CNN/DM SQuAD CelebA CIFAR IWSLT
Benchmarks: ImageNet Cityscapes CNN/DM SQuAD CelebA IWSLT
Metrics: empirical test risk average exit layer relative improvement mIoU ROUGE-L Token-F1 LPIPS BLEU FLOPS
Baselines: LTT (as used in CALM) empirical threshold CALM
Builds on: Conformal Risk Control Learn-then-Test CALM Waudby-Smith-Ramdas bound
Closest papers by meaning
- Beyond Greedy Exits: Improved Early Exit Decisions for Risk Control and Reliability (NeurIPS 2025 · similarity 0.90)
- Jointly-Learned Exit and Inference for a Dynamic Neural Network (ICLR 2024 · similarity 0.88)
- How to Train Your Multi-Exit Model? Analyzing the Impact of Training Strategies (ICML 2025 · similarity 0.86)
- BEEM: Boosting Performance of Early Exit DNNs using Multi-Exit Classifiers as Experts (ICLR 2025 · similarity 0.85)
- Deep Neural Networks Tend To Extrapolate Predictably (ICLR 2024 · similarity 0.84)
- When to Grow? A Fitting Risk-Aware Policy for Layer Growing in Deep Neural Networks (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.