Research map / Architectures and efficiency / Quantization
Binary Quadratic Quantization: Beyond First-Order Quantization for Real-Valued Matrix Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Binary Quadratic Quantization (BQQ), which approximates real-valued matrices as sums of products of two binary matrices with independent scaling factors, optimized via AMFD-based PUBO solving, and evaluates on matrix compression and ViT/LLM post-training quantization.
Problem
First-order scalar quantization (UQ, BCQ) has very limited expressiveness at ultra-low bit widths, and existing factorization-based compression either uses real-valued factors or lacks fully binary, per-matrix-scaled decompositions of real-valued matrices.
Approach
Represent a real matrix as a sum of scaled binary matrix products (r_i Y_i Z_i plus rank-one-like bias terms s_i Y_i 1 and t_i 1 Z_i plus a global bias), generalizing BCQ; solve the mixed-integer subproblem by reducing it to a PUBO solved with an annealed mean-field descent (AMFD) extended to general PUBO, alternating with closed-form least squares for scalars.
Key results
- On ImageNet with pseudo 2-bit data-free PTQ, BQQ reaches 58.25% (DeiT-S) and 72.09% (DeiT-B) top-1 versus 3.23% and 2.45% for UQ.
- With calibration, c-BQQ at pseudo 2-bit gets 69.41% on DeiT-S and 77.94% on DeiT-B, outperforming listed PTQ baselines such as COMQ (67.19%, 77.14%) on those models.
- On matrix compression, BQQ shows a better MSE vs. memory trade-off than UQ, BCQ, VQ, SVD and E8 LVQ on most tested matrices, though JPEG is better on ImageNet image data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use binary quadratic matrix-product quantization instead of first-order BCQ | ViT weights · ImageNet, data-free 2-bit; DeiT-S | top-1 accuracy: 10.83 -> 58.25 | Helped |
| Use binary quadratic matrix-product quantization instead of first-order BCQ | ViT weights · ImageNet, data-free 2-bit; DeiT-B | top-1 accuracy: 12.99 -> 72.09 | Helped |
| Use binary quadratic matrix-product quantization instead of first-order BCQ | ViT weights · ImageNet, data-free 2-bit; Swin-T | top-1 accuracy: 18.62 -> 57.37 | Helped |
| Use binary quadratic matrix-product quantization instead of first-order BCQ | ViT weights · ImageNet, data-free 2-bit; Swin-S | top-1 accuracy: 34.84 -> 68.17 | Helped |
| Use BQQ instead of the strongest listed PTQ baseline | DeiT-S weights with calibration · ImageNet, calibration-based 2-bit; DeiT-S | top-1 accuracy: 67.19 -> 69.41 | Helped |
| Use BQQ instead of the strongest listed PTQ baseline | DeiT-B weights with calibration · ImageNet, calibration-based 2-bit; DeiT-B | top-1 accuracy: 77.14 -> 77.94 | Helped |
| Use group-wise c-BQQ instead of column-wise COMQ | Swin-T weights with calibration · ImageNet, calibration-based 2-bit; Swin-T | top-1 accuracy: 74.05 -> 74.03 | No clear effect |
| Use group-wise c-BQQ instead of column-wise COMQ | Swin-S weights with calibration · ImageNet, calibration-based 2-bit; Swin-S | top-1 accuracy: 78.02 -> 78.47 | Helped |
| Increase optimization steps beyond 50,000 (ablation) | BQQ on DeiT-S · ImageNet, data-free pseudo 2-bit | top-1 accuracy: +1.99% | Helped |
| Increase optimization steps beyond 50,000 (ablation) | BQQ on DeiT-B · ImageNet, data-free pseudo 2-bit | top-1 accuracy: +0.82% | Helped |
| Fine-tune continuous parameters after BQQ | Qwen2.5-0.5B · WikiText-2, 2-bit | perplexity: 17106.9 -> 49.5 | Helped |
| Fine-tune continuous parameters after BQQ | Qwen2.5-1.5B · WikiText-2, 2-bit | perplexity: 688.3 -> 23.6 | Helped |
Surprising findings
- Data-free BQQ at pseudo 2-bit outperforms calibration-free-comparable baselines by large margins (e.g., 58.25% vs 10.83% for BCQ on DeiT-S).
- On the 0.5B Qwen2.5 model, BQQ without calibration collapses (PPL 17106.9 at 2-bit), while t-BQQ recovers much of the performance.
Limitations
- Greedy, layer-wise and stage-wise optimization is suboptimal relative to joint optimization of all binary matrices and scalars.
- Quantization process has non-negligible computational cost (e.g., 13 to 57 minutes for DeiT models).
- No theoretical guarantee that BQQ outperforms first-order quantization, only an upper bound on approximation error; BQQ can degrade on smaller LLMs like Qwen2.5-0.5B.
Experimental setup
Datasets: ImageNet WikiText-2 TSPLIB SIFT Gaussian random matrix
Benchmarks: ImageNet WikiText-2 PIQA WinoGrande ARC-Easy ARC-Challenge HellaSwag BoolQ
Metrics: MSE memory size top-1 accuracy perplexity average accuracy
Baselines: UQ BCQ SVD VQ E8 LVQ JPEG COMQ FQ-ViT PTQ4ViT RepQ-ViT ERQ PSAQ-ViT
Builds on: Binary Coding Quantization Annealed Mean Field Descent GPTQ
Closest papers by meaning
- FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation (CVPR 2025 · similarity 0.90)
- 2DQuant: Low-bit Post-Training Quantization for Image Super-Resolution (NeurIPS 2024 · similarity 0.88)
- QBB: Quantization with Binary Bases for LLMs (NeurIPS 2024 · similarity 0.87)
- Image and Video Tokenization with Binary Spherical Quantization (ICLR 2025 · similarity 0.87)
- GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration (ICML 2025 · similarity 0.86)
- AffineQuant: Affine Transformation Quantization for Large Language Models (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.