Research map / Architectures and efficiency / Quantization
RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformers
Zhikai Li, Junrui Xiao, Lianwei Yang, Qingyi Gu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| reparameterize channel-wise to layer-wise quantization for post-LayerNorm activations (vs direct layer-wise) (ablation) | RepQ-ViT · ImageNet W4/A4 | Top-1 accuracy: 33.17 -> 69.03 (DeiT-S) | Improved |
| reparameterize channel-wise to layer-wise for post-LayerNorm on Swin-S (ablation) | RepQ-ViT · ImageNet W4/A4 | Top-1 accuracy: 57.63 -> 79.45 | Improved |
| use scale reparameterization instead of true channel-wise quantization (ablation) | RepQ-ViT · ImageNet W4/A4 | Top-1 accuracy: 70.28 -> 69.03 | Hurt |
| use log sqrt2 quantizer instead of log2 for post-Softmax (ablation) | RepQ-ViT · ImageNet W4/A4 | Top-1 accuracy: 77.87 -> 79.45 (Swin-S) | Improved |
| use log sqrt2 quantizer instead of log2 on DeiT-S (ablation) | RepQ-ViT · ImageNet W4/A4 | Top-1 accuracy: 67.71 -> 69.03 | Improved |
| reparameterize log sqrt2 to log2 for inference (ablation) | RepQ-ViT post-Softmax · ImageNet W4/A4 | Top-1 accuracy: 69.03 -> 69.03 | No clear effect |
| apply RepQ-ViT instead of APQ-ViT for 4-bit quantization | ViT-B · ImageNet W4/A4 | Top-1 accuracy: 41.41 -> 68.48 | Improved |
| apply RepQ-ViT instead of APQ-ViT in COCO W4/A4 with Swin-S Mask R-CNN | Mask R-CNN Swin-S · COCO | AP box: 44.7 -> 44.2 | Hurt |
| apply RepQ-ViT instead of APQ-ViT in COCO W4/A4 with Swin-T Cascade Mask R-CNN | Cascade Mask R-CNN Swin-T · COCO | AP box: 27.2 -> 47.0 | Improved |
| avoid reconstruction and hyperparameters; use RepQ-ViT instead of PTQ4ViT | DeiT-S W4/A4 · ImageNet | GPU minutes: 3.2 -> 1.3 | Improved |
Later papers that compared with this method
15 later accepted papers report RepQ-ViT as a baseline in their results tables; 15 of them report a better number on most of the results they share with it.
- SPA-Q: Structure-Preserving Adaptive Post-Training Quantization for Monocular Depth Estimation (NeurIPS 2026 · better on 4 of 4 shared results)
- UQ-ViT: Harmonizing Extreme Activations with Hardware-Friendly Uniform Quantization in Vision Transformers (AAAI 2026 · better on 3 of 3 shared results)
- Quantized Visual Geometry Grounded Transformer (ICLR 2026 · better on 1 of 1 shared result)
- QuEPT: Quantized Elastic Precision Transformers with One-Shot Calibration for Multi-Bit Switching (AAAI 2026 · better on 1 of 1 shared result)
- Quantization without Tears (CVPR 2025 · better on 9 of 9 shared results)
- AIQViT: Architecture-Informed Post-Training Quantization for Vision Transformers (AAAI 2025 · better on 7 of 7 shared results)
- I&S-ViT: An Inclusive & Stable Method for Post-Training ViTs Quantization (TPAMI 2025 · better on 7 of 7 shared results)
- APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformers (CVPR 2025 · better on 3 of 3 shared results)
7 more not listed.
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- PTQ4ViT — PTQ4ViT: Post-Training Quantization for Vision Transformers with Twin Uniform Quantization (ECCV 2022) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformers (CVPR 2025 · similarity 0.93)
- AIQViT: Architecture-Informed Post-Training Quantization for Vision Transformers (AAAI 2025 · similarity 0.93)
- ERQ: Error Reduction for Post-Training Quantization of Vision Transformers (ICML 2024 · similarity 0.92)
- FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation (CVPR 2025 · similarity 0.92)
- GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers (NeurIPS 2025 · similarity 0.92)
- VLM-PTQ: Efficient Post-Training Quantization for Large Vision-Language Models (CVPR 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
