Research map / Architectures and efficiency / RNNs and normalization
QUTE: Quantifying Uncertainty in TinyML models with Early-exit-assisted ensembles for model-monitoring
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes QUTE, an early-exit-assisted ensemble that attaches lightweight extra output heads to a tinyML network's final exit and distills early-exit knowledge into them via early-view assistance, enabling single-pass uncertainty quantification and accuracy-drop detection on MCUs.
Problem
Uncertainty quantification for model monitoring on KB-sized tinyML devices is impractical because existing ensemble and early-exit ensemble methods impose large memory and compute overheads, and corrupted-in-distribution inputs often cause overconfident predictions.
Approach
Append K lightweight output heads (single depthwise-conv layer each) at the final exit and train them with early-view assistance, where each head's learning layer is weight-copied from a temporary early-exit head before every batch; early-exits are discarded at inference and predictions are averaged over the heads.
Key results
- On Big-MCU, QUTE reduces latency by 31% on average vs EE-ensemble and has 58% smaller models than EE-ensemble while matching its accuracy.
- QUTE averages 59% smaller model size than EE-ensemble across datasets, with comparable calibration on tiny models (e.g., MNIST NLL 0.199 vs 0.266 for EE-ensemble).
- QUTE achieves the best average AUPRC for accuracy-drop detection across MNIST-C, SpeechCmd-C, CIFAR10-C and TinyImageNet-C, outperforming all listed baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use QUTE instead of the strongest listed baseline | 4-layer CNN · MNIST ID | NLL: 0.227$\pm$0.002 -> 0.199$\pm$0.010 | Helped |
| Use QUTE instead of DEEP | DSCNN · SpeechCmd ID | NLL: 0.205$\pm$0.012 -> 0.202$\pm$0.016 | No clear effect |
| Add two depthwise-separable CONV layers to QUTE output blocks (ablation) | QUTE on Resnet-8 · CIFAR10 ID | F1: 0.858$\pm$0.001 -> 0.878$\pm$0.003 | Helped |
| Add a dense layer to QUTE output blocks (ablation) | QUTE on MobileNetV2 · TinyImagenet ID | F1: 0.395$\pm$0.014 -> 0.381$\pm$0.010 | Hurt |
| Add EV weight transfer (ablation) | QUTE on Resnet-8 · CIFAR10 ID | NLL: 0.459$\pm$0.006 -> 0.435$\pm$0.007 | Helped |
| Add EV weight transfer (ablation) | QUTE on MobileNetV2 · TinyImagenet ID | NLL: 4.743$\pm$0.111 -> 3.813$\pm$0.105 | Helped |
| Remove additional FC layers from EE-ensemble early exits (ablation) | EE-ensemble on Resnet-8 · CIFAR10 ID | F1: 0.85 -> 0.78 | Hurt |
| Remove additional FC layers from EE-ensemble early exits (ablation) | EE-ensemble on Resnet-8 · CIFAR10 CID | NLL: 1.256 -> 1.459 | Hurt |
| Exclude original final-exit prediction from the EE-ensemble mean (ablation) | EE-ensemble on Resnet-8 · CIFAR10 ID | F1: 0.854 -> 0.818 | Hurt |
| Apply temperature scaling (ablation) | QUTE on 4-layer CNN · MNIST-C | AUPRC: 0.63 -> 0.52 | Hurt |
| Use QUTE instead of the strongest listed baseline | 4-layer CNN · MNIST ID | ID×ID error | AUROC: 0.85 -> 0.87 | Helped |
| Use QUTE instead of the strongest listed baseline | Resnet-8 · CIFAR10 ID | ID×ID error | AUROC: 0.87 -> 0.86 | Hurt |
Surprising findings
- Smaller networks were found to be less overconfident on corrupted inputs; the 4-stack ResNet was worst on CID despite best ID calibration.
- G-ODIN, a specialized OOD detector, performed poorly on MNIST and SpeechCmd failure detection, even below BASE.
Limitations
- Ensemble size K is bounded by base network depth, and increasing K beyond a point degrades the shared base network's calibration.
- On large models (MobileNetV2 on TinyImageNet), QUTE needs extra learning layers to stay competitive; training is slower due to weight transfer.
- Performance on OOD detection is weaker than specialized OOD detectors for large models.
Experimental setup
Datasets: MNIST SpeechCommands CIFAR10 TinyImageNet MNIST-C CIFAR10-C TinyImageNet-C SpeechCmd-C FashionMNIST SVHN
Benchmarks: MNIST-C SpeechCmd-C CIFAR10-C TinyImageNet-C FashionMNIST OOD SVHN OOD SpeechCmd-OOD
Metrics: AUPRC AUROC F1 Brier score NLL ECE model size (KB) latency (ms)
Baselines: BASE Monte Carlo Dropout DeepEnsembles EE-ensemble HYDRA G-ODIN Postnets temperature scaling
Builds on: EE-ensemble early-view assistance Deep Ensembles HYDRA Monte Carlo Dropout
Closest papers by meaning
- AtomNet: Designing Tiny Models from Operators Under Extreme MCU Constraints (AAAI 2025 · similarity 0.84)
- TinyTTA: Efficient Test-time Adaptation via Early-exit Ensembles on Edge Devices (NeurIPS 2024 · similarity 0.84)
- Integrating Local and Global Entropy for Uncertainty Quantification in LLMs (NeurIPS 2026 · similarity 0.83)
- Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space Models (ICML 2025 · similarity 0.83)
- Uncertainty Quantification with the Empirical Neural Tangent Kernel (NeurIPS 2025 · similarity 0.83)
- Measuring Stochastic Data Complexity with Boltzmann Influence Functions (ICML 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.