Research map / Architectures and efficiency / Attention and transformers
Semantic Alignment and Reinforcement for Data-Free Quantization of Vision Transformers
Yunshan Zhong, Yuyao Zhou, Yuxin Zhang, Wanchen Sui, Shen Li, Yong Li, Fei Chao, Rongrong Ji
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Attention Priors Alignment to one-hot baseline (ablation) | SARDFQ · W4/A4 DeiT-S ImageNet | top-1 accuracy: 51.73 -> 60.26 | Improved |
| add Multi-Semantic Reinforcement alone to one-hot baseline (ablation) | SARDFQ · W4/A4 DeiT-S ImageNet | top-1 accuracy: 51.73 -> 50.75 | Hurt |
| add Softlabel Learning alone to one-hot baseline (ablation) | SARDFQ · W4/A4 DeiT-S ImageNet | top-1 accuracy: 51.73 -> 52.02 | Improved |
| add Softlabel Learning to MSR (ablation) | SARDFQ · W4/A4 DeiT-S ImageNet | top-1 accuracy: 50.75 -> 56.08 | Improved |
| add MSR to APA (ablation) | SARDFQ · W4/A4 DeiT-S ImageNet | top-1 accuracy: 60.26 -> 61.58 | Improved |
| add MSR to APA+SL (full method) (ablation) | SARDFQ · W4/A4 DeiT-S ImageNet | top-1 accuracy: 60.51 -> 62.29 | Improved |
| add SL to APA+MSR (full method) (ablation) | SARDFQ · W4/A4 DeiT-S ImageNet | top-1 accuracy: 61.58 -> 62.29 | Improved |
| apply APA only to deeper blocks (S=L/2) instead of all blocks (ablation) | SARDFQ · W4/A4 DeiT-S ImageNet | top-1 accuracy: 61.96 -> 62.29 | Improved |
| add depth weighting l/L to APA loss (ablation) | SARDFQ · W4/A4 DeiT-S ImageNet | top-1 accuracy: 61.32 -> 62.29 | Improved |
| use real attention priors instead of GMM priors (ablation) | SARDFQ · W4/A4 DeiT-S ImageNet | top-1 accuracy: 62.29 -> 63.19 | Improved |
| use Laplace instead of GMM attention priors (ablation) | SARDFQ · W4/A4 DeiT-S ImageNet | top-1 accuracy: 62.29 -> 62.16 | No clear effect |
| set K_APA to 5 instead of 1 (ablation) | SARDFQ · W4/A4 DeiT-S ImageNet | top-1 accuracy: 61.13 -> 62.29 | Improved |
Closest papers by meaning
- MimiQ: Low-Bit Data-Free Quantization of Vision Transformers with Encouraging Inter-Head Attention Similarity (AAAI 2025 · similarity 0.90)
- AIQViT: Architecture-Informed Post-Training Quantization for Vision Transformers (AAAI 2025 · similarity 0.86)
- RPFQ-ViT: Rotated Phase-Frame Quantization for Extremely Low-Bit Weights in Vision Transformers (NeurIPS 2026 · similarity 0.86)
- Noise Diffusion for Enhancing Semantic Faithfulness in Text-to-Image Synthesis (CVPR 2025 · similarity 0.85)
- FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation (CVPR 2025 · similarity 0.85)
- PTQ4DiT: Post-training Quantization for Diffusion Transformers (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
