Research map / Architectures and efficiency / Quantization
Amortized-Precision Quantization for Early-Exit Vision Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Amortized-Precision Quantization (APQ) and MAQEE, a bi-level risk-controlled method that jointly optimizes early-exit thresholds and layer-wise bit-widths for quantized ViTs, with self-distillation recovery.
Problem
Low-precision quantization of Vision Transformers combined with early exiting is fragile: quantization noise perturbs exit confidence, causing premature or delayed exits, while static FPQ/MPQ methods assume full-depth execution.
Approach
Weight each layer's bit-cost and sensitivity by its utilization probability under early exiting (APQ), and alternate between outer exit-threshold search and inner bit-width reallocation guided by a combined risk score (PGR, BSR, QID, SQNR^-1) plus self-distillation recovery.
Key results
- On ImageNet with Swin under W4A4, MAQEE reaches 80.78% accuracy at 2.12E11 BOPs versus 1.39E13 BOPs for FP32 (83.50%), a large BOPs reduction.
- On SceneParse150 with ViT-B, MAQEE achieves 31.20 IoU versus 34.14 for FP32 and 27.34 for ViT-EE+ERQ.
- On MS COCO with ViT-B, MAQEE reaches 50.83 mAP@0.5 versus 40.42 for ViT-EE+ERQ and 68.23 for FP32.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add PGR risk term to MAQEE objective (ablation) | MAQEE · ImageNet | Acc.: 79.41 vs 61.18 without PGR | Helped |
| add BSR risk term (ablation) | MAQEE · ImageNet | Acc.: 79.41 vs 76.53 without BSR | Helped |
| add SQNR^-1 risk term (ablation) | MAQEE · ImageNet | Acc.: 79.41 vs 78.35 without SQNR^-1 | Helped |
| add QID risk term (ablation) | MAQEE · ImageNet | Acc.: 79.41 vs 79.06 without QID | Helped |
| MAQEE vs strongest quantized EE baselines (ViT-B) | MAQEE · ImageNet | Acc. at standard config: up to ~20% over strong baselines (paper claim) | Helped |
| MAQEE with adaptive thresholds and bit-widths vs FPQ/MPQ with early exit | ViT-B on SceneParse150 and MS COCO · SceneParse150 / MS COCO | IoU / mAP@0.5: 31.20 vs 27.34 (IoU); 50.83 vs 41.04 (mAP@0.5) | Helped |
| Quantization under early exiting on object detection | ViT-B with MAQEE · MS COCO | mAP@0.5: 50.83 vs FP32 68.23 | Hurt |
Surprising findings
- Quantizing EE-trained models can cause up to a 50% accuracy drop, and applying EE to already quantized models makes exit-head training difficult.
- Removing PGR from MAQEE drops standard-config ImageNet-style accuracy to 61.18% on the ablation table while reducing BOPs.
Limitations
- Object detection still shows a noticeable accuracy drop under early exit plus quantization.
- The NP-hardness proof and the claimed monotonic convergence rely on exact coordinate solves over discretized candidate sets, which is not achieved in practice.
Experimental setup
Datasets: ImageNet CIFAR-100 SceneParse150 MS COCO
Benchmarks: ImageNet CIFAR-100 SceneParse150 MS COCO
Metrics: top-1 accuracy mean IoU mAP@0.5 BOPs average exit layer
Baselines: ViT-EE LGViT RepQ ERQ MPTQ
Builds on: early exiting mixed-precision quantization knowledge distillation
Closest papers by meaning
- AIQViT: Architecture-Informed Post-Training Quantization for Vision Transformers (AAAI 2025 · similarity 0.90)
- Bi-ViT: Pushing the Limit of Vision Transformer Quantization (AAAI 2024 · similarity 0.88)
- RPFQ-ViT: Rotated Phase-Frame Quantization for Extremely Low-Bit Weights in Vision Transformers (NeurIPS 2026 · similarity 0.88)
- GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers (NeurIPS 2025 · similarity 0.88)
- GradQ-ViT: Robust and Efficient Gradient Quantization for Vision Transformers (AAAI 2025 · similarity 0.88)
- APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformers (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.