Research map / Architectures and efficiency / Quantization
AIQViT: Architecture-Informed Post-Training Quantization for Vision Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AIQViT, a post-training quantization method for ViTs that adds NAS-selected low-rank weight compensation (with curriculum-learning reconstruction) and a dynamic focusing quantizer for post-Softmax activations.
Problem
Post-training quantization of vision transformers suffers large accuracy loss at low bit-widths because weight quantization error is underestimated and log2 quantizers for post-Softmax activations waste resolution near zero.
Approach
Learnable low-rank weights added to every linear layer compensate weight-quantization error, with the rank chosen by differentiable architecture search; a Dynamic Focusing Quantizer learns a [b1,b2] interval for post-Softmax activations and uses uniform quantization inside it instead of log2 quantization.
Key results
- On ImageNet W3/A3, AIQViT reaches 38.51% top-1 on DeiT-T versus 15.68% for QDrop and 1.15% for RepQ-ViT.
- On ImageNet W4/A4, AIQViT reaches 62.33% on DeiT-T and 72.75% on DeiT-S, above RepQ-ViT (57.43%, 69.03%).
- On ModelNet40 W4/A4, AIQViT achieves 82.66 OA, about 5 points above BRECQ.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AIQViT instead of QDrop | DeiT-T quantization · ImageNet, W3/A3 | top-1 accuracy: 15.68 -> 38.51 | Helped |
| Use AIQViT instead of RepQ-ViT | DeiT-T quantization · ImageNet, W4/A4 | top-1 accuracy: 57.43 -> 62.33 | Helped |
| Use AIQViT instead of RepQ-ViT | DeiT-T quantization · ImageNet, W6/A6 | top-1 accuracy: 70.76 -> 70.78 | No clear effect |
| Use AIQViT instead of BRECQ | Mask R-CNN with Swin-T · COCO, W4/A4 | box AP: 33.7 -> 38.2 | Helped |
| Use AIQViT instead of APQ-ViT | Mask R-CNN with Swin-S · COCO, W4/A4 | box AP: 44.7 -> 44.1 | Hurt |
| Use AIQViT instead of RepQ-ViT | Point Transformer · ModelNet40, W4/A4 | OA: 77.98 -> 82.66 | Helped |
| Use AIQViT instead of RepQ-ViT | Point Transformer · ShapeNetPart, W4/A4 | c.mIoU: 65.78 -> 68.27 | Helped |
| Add AILoC (ablation) | AIQViT · ImageNet, DeiT-T, W3/A3 | top-1 accuracy: 23.28 -> 38.51 | Helped |
| Add DFQ (ablation) | AIQViT · ImageNet, DeiT-T, W3/A3 | top-1 accuracy: 26.58 -> 38.51 | Helped |
| Add curriculum learning (ablation) | AIQViT · ImageNet, DeiT-T, W3/A3 | top-1 accuracy: 38.01 -> 38.51 | Helped |
| Add AILoC (ablation) | AIQViT · ImageNet, DeiT-T, W4/A4 | top-1 accuracy: 56.39 -> 62.33 | Helped |
| Add DFQ (ablation) | AIQViT · ImageNet, DeiT-T, W4/A4 | top-1 accuracy: 57.23 -> 62.33 | Helped |
Surprising findings
- Increasing the low-rank size r does not guarantee better accuracy; for DeiT-S at W4/A4, r=20 outperforms r=100 by 1.0%.
Limitations
- Sensitive to calibration set size: only 256 samples causes overfitting and subpar results at W3/A3 and W4/A4.
- Low-bit (W3/A3) accuracy remains far below full precision on several models.
Experimental setup
Datasets: ImageNet ModelNet40 ShapeNetPart COCO
Benchmarks: ImageNet COCO ModelNet40 ShapeNetPart
Metrics: top-1 accuracy box AP mask AP mAcc OA c.mIoU i.mIoU
Baselines: RepQ-ViT BRECQ QDrop PTQ4ViT APQ-ViT Min-Max
Builds on: LoRA BRECQ QDrop RepQ-ViT DARTS
Closest papers by meaning
- ERQ: Error Reduction for Post-Training Quantization of Vision Transformers (ICML 2024 · similarity 0.92)
- APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformers (CVPR 2025 · similarity 0.92)
- GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers (NeurIPS 2025 · similarity 0.92)
- Instance-Aware Group Quantization for Vision Transformers (CVPR 2024 · similarity 0.91)
- GradQ-ViT: Robust and Efficient Gradient Quantization for Vision Transformers (AAAI 2025 · similarity 0.91)
- FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation (CVPR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.