Research map / Architectures and efficiency / Quantization
MixA-Q: Revisiting Activation Sparsity for Vision Transformers from a Mixed-Precision Quantization Perspective
Weitian Wang, Rai Shubham, Cecilia De La Parra, Akash Kumar
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add sparsity-aware quantization adaptation (MixA-Q at full 4-bit) (ablation) | OFQ W4A4 Mask R-CNN Swin-T · COCO det | mAP: 43.1 -> 43.8 | Improved |
| use MixA-Q instead of uniform 3-bit activations | OFQ W4A4 · COCO det, 1.35x | mAP: 41.4 -> 42.3 | Improved |
| use MixA-Q instead of SparseViT at 1.24x | OFQ W4A4 · COCO det, 1.24x | mAP: 43.1 -> 43.2 | No clear effect |
| add 30% activation pruning to MixA-Q | MixA-Q · COCO det, 1.53x | mAP: 41.4 -> 42.1 | Improved |
| add 30% activation pruning to MixA-Q | MixA-Q · COCO det, 1.82x | mAP: 40.1 -> 40.5 | Improved |
| use uniform-sum compression ratio sampling instead of naive uniform sampling (ablation) | SAQA · COCO det, 3.36 bit | mAP: 43.0 -> 43.4 | Improved |
| use uniform-sum compression ratio sampling instead of naive uniform sampling (ablation) | SAQA · COCO det, 3 bit | mAP: 41.9 -> 42.4 | Improved |
| start SAQA from the W4A4-quantized model rather than floating point (ablation) | SAQA · COCO det, 3.36 bit | mAP: 42.8 -> 43.4 | Improved |
| allow elevating compressed windows back to high precision in later blocks (ablation) | MixA-Q · COCO det | mAP: -0.1% | Hurt |
| use MixA-Q instead of SparseViT on out-of-distribution weather images | OFQ W4A4 · COCO-O | mAP_o: 29.6 -> 30.5 | Improved |
| use MixA-Q instead of SparseViT under worst window selection | OFQ W4A4 · COCO worst window selection | mAP_w: 24.0 -> 28.8 | Improved |
| use MixA-Q instead of SparseViT on clean COCO at 1.4x | OFQ W4A4 · COCO det, 1.4x | mAP: 42.4 -> 41.5 | Hurt |
Closest papers by meaning
- AIQViT: Architecture-Informed Post-Training Quantization for Vision Transformers (AAAI 2025 · similarity 0.89)
- GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers (NeurIPS 2025 · similarity 0.88)
- ERQ: Error Reduction for Post-Training Quantization of Vision Transformers (ICML 2024 · similarity 0.88)
- Amortized-Precision Quantization for Early-Exit Vision Transformers (NeurIPS 2026 · similarity 0.88)
- FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation (CVPR 2025 · similarity 0.88)
- DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformers (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
