Research map / Detection and segmentation / CNN features
PTQ4SAM: Post-Training Quantization for Segment Anything
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PTQ4SAM, a post-training quantization framework for the Segment Anything Model that removes post-Key-Linear bimodal distributions via a sign-based equivalent transformation and uses an adaptive power-of-two-base log quantizer for post-Softmax attention.
Problem
Post-training quantization of Segment Anything Model (SAM) is hampered by bimodal post-Key-Linear activations and highly varied post-Softmax distributions across self- and cross-attention, causing large accuracy loss at low bit-width.
Approach
A Bimodal Integration strategy multiplies query and key linears by a channel-wise sign factor so the bimodal key distribution becomes normal (mathematically equivalent, absorbed offline), and an Adaptive Granularity Quantization searches a power-of-two log base (tau in {2^0,2^1,2^2}) per post-Softmax quantizer by minimizing the error of the attention-value product.
Key results
- PTQ4SAM-L at W6A6 on SAM-L with H-Deformable-DETR reaches 41.2% mAP vs 41.5% full precision on COCO instance segmentation.
- At W4A4 on SAM-L with DINO, PTQ4SAM-L reaches 36.6% mAP versus QDrop's 27.5%.
- 6-bit SAM-L achieves theoretical 3.9x FLOPs reduction and 4.9x storage savings with near-lossless accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Bimodal Integration and Adaptive Granularity Quantization (ablation) | PTQ4SAM on SAM-L · COCO instance segmentation | mAP: 25.8 -> 32.1 | Helped |
| add Bimodal Integration (ablation) | PTQ4SAM on SAM-L · COCO instance segmentation, W4A4 | mAP: 25.8 -> 29.2 | Helped |
| add Adaptive Granularity Quantization (ablation) | PTQ4SAM on SAM-L · COCO instance segmentation, W4A4 | mAP: 25.8 -> 27.3 | Helped |
| replace uniform post-Softmax quantization with AGQ (ablation) | SAM-L · COCO instance segmentation, W4A4 | mAP: 25.3 -> 27.8 | Helped |
| replace Log2 post-Softmax quantization with AGQ (ablation) | SAM-L · COCO instance segmentation, W4A4 | mAP: 26.5 -> 27.8 | Helped |
| replace uniform post-Softmax quantization with AGQ (ablation) | SAM-B · COCO instance segmentation, W4A4 | mAP: 13.3 -> 15.0 | Helped |
| replace Log2 post-Softmax quantization with AGQ (ablation) | SAM-H · COCO instance segmentation, W4A4 | mAP: 37.3 -> 37.6 | Helped |
Surprising findings
- Bimodal distribution appears in post-Key-Linear activations but not in post-LayerNorm activations, so prior LayerNorm-based channel equalization does not apply.
- Naively applying Log2 quantization to SAM attention scores is unstable, sometimes performing worse than uniform quantization (e.g., W6A6 on SAM-B).
Limitations
- The root cause of the bimodal distribution in SAM remains unclear.
- Speedups are theoretical FLOP-based estimates rather than measured hardware latency.
- W4A4 accuracy still lags full precision substantially for some detectors.
Experimental setup
Datasets: SA-1B MS-COCO ADE20K DOTA-v1.0
Benchmarks: MS-COCO ADE20K DOTA-v1.0
Metrics: mAP mIoU FLOPs storage
Baselines: MinMax Percentile OMSE AdaRound BRECQ QDrop Log2 quantizer Uniform quantizer FastSAM
Builds on: OMSE QDrop AdaRound BRECQ
Closest papers by meaning
- TinySAM: Pushing the Envelope for Efficient Segment Anything Model (AAAI 2025 · similarity 0.90)
- AIQViT: Architecture-Informed Post-Training Quantization for Vision Transformers (AAAI 2025 · similarity 0.88)
- Instance-Aware Group Quantization for Vision Transformers (CVPR 2024 · similarity 0.88)
- Fine-Grained Post-Training Quantization for Large Vision Language Models with Quantization-Aware Integrated Gradients (CVPR 2026 · similarity 0.87)
- UncertainSAM: Fast and Efficient Uncertainty Quantification of the Segment Anything Model (ICML 2025 · similarity 0.87)
- VLM-PTQ: Efficient Post-Training Quantization for Large Vision-Language Models (CVPR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.