Research map / Architectures and efficiency / RNNs and normalization
BEEM: Boosting Performance of Early Exit DNNs using Multi-Exit Classifiers as Experts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BEEM, an early-exit criterion that treats each intermediate exit as an expert, aggregating consistent confidence scores with layer weights and setting thresholds from exit error rates, giving 1.5-2.1x speed-up on GLUE and COCO captioning.
Problem
Early-exit DNNs rely on exit decision criteria (confidence or patience based) that either ignore information from previous exits or treat all exits equally, limiting the accuracy-latency trade-off.
Approach
Treat each intermediate exit classifier as a weighted expert; accumulate its confidence with previous exits only while consecutive predictions agree (resetting on disagreement), and exit when the aggregated score exceeds a threshold. Thresholds are chosen by a constrained search so each exit's error rate is below the final layer's error rate.
Key results
- On GLUE with ALBERT, BEEM-A reaches 1.91x-2.09x speed-up on dev set with accuracy gains over the final layer on SST-2 (+0.4), RTE (+0.7) and QQP (+0.5).
- On COCO captioning, BEEM-C achieves 1.71x speed-up with CIDEr 145.1 versus 147.1 for the final exit.
- On ALBERT-Large, BEEM-A achieves up to 3.33x speed-up on QQP with +0.1 accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use accuracy-based rather than cost-based exit weights | BEEM on GLUE · GLUE dev and test tasks | accuracy: BEEM-A consistently outperforms BEEM-C for all the datasets in terms of accuracy | Mixed |
| Use accuracy-based rather than cost-based exit weights | BEEM on GLUE · GLUE dev and test tasks | speedup | Mixed |
| Use BEEM-A early exit instead of final-layer inference | ALBERT · GLUE tasks | accuracy: +0.4 on SST-2 test; -0.3 on MNLI test; +0.6 on RTE test | Mixed |
| Use BEEM-A early exit instead of final-layer inference | BERT · CoLA | accuracy: +0.3 | Helped |
| Use BEEM-A early exit instead of final-layer inference | Swin Transformer + GPT-2 image captioner · COCO Karpathy test | CIDEr: 147.1 -> 146.5 | Hurt |
| Use threshold selection constrained by the final classifier's error rate (ablation) | BEEM-A on ALBERT-Base · SST-2 | accuracy: 92.4 -> 92.6 | Helped |
| Use threshold selection constrained by the final classifier's error rate (ablation) | BEEM-A on ALBERT-Base · QQP | accuracy: 90.4 -> 91.1 | Helped |
| Use threshold selection constrained by the final classifier's error rate (ablation) | BEEM-A on ALBERT-Base · SST-2 | speedup: 1.81x -> 1.98x | Helped |
| Use threshold selection constrained by the final classifier's error rate (ablation) | BEEM-A on ALBERT-Base · QQP | speedup: 2.05x -> 2.09x | Helped |
| Use threshold selection constrained by the final classifier's error rate (ablation) | BEEM-A on ALBERT-Large · SST-2 | accuracy: 93.1 -> 93.4 | Helped |
| Use threshold selection constrained by the final classifier's error rate (ablation) | BEEM-A on ALBERT-Large · QQP | accuracy: 91.1 -> 91.3 | Helped |
| Use threshold selection constrained by the final classifier's error rate (ablation) | BEEM-A on ALBERT-Large · SST-2 | speedup: 2.19x -> 2.31x | Helped |
Surprising findings
- BEEM-A sometimes exceeds the final layer's accuracy on easier GLUE tasks, which the authors attribute to reduced overthinking.
Limitations
- Accuracy drops on the harder image captioning task relative to the final layer.
- Thresholds tuned on the validation set may not generalize to the test set.
- BEEM-C's cost weights are assumed proportional to accuracy, which the authors note does not hold due to overthinking.
Experimental setup
Datasets: GLUE COCO
Benchmarks: GLUE COCO
Metrics: accuracy speed-up BLEU-1 BLEU-4 METEOR CIDEr SPICE ROUGE-L
Baselines: DeeBERT ElasticBERT FastBERT PABEE LeeBERT ZTW PCEEBERT MuE PALBERT JEI-DNN DeeCap
Builds on: PABEE DeeBERT BranchyNet ALBERT BERT
Closest papers by meaning
- Beyond Greedy Exits: Improved Early Exit Decisions for Risk Control and Reliability (NeurIPS 2025 · similarity 0.85)
- Fast yet Safe: Early-Exiting with Risk Control (NeurIPS 2024 · similarity 0.85)
- Jointly-Learned Exit and Inference for a Dynamic Neural Network (ICLR 2024 · similarity 0.84)
- How to Train Your Multi-Exit Model? Analyzing the Impact of Training Strategies (ICML 2025 · similarity 0.84)
- EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism (ICML 2024 · similarity 0.83)
- Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.