Research map / Architectures and efficiency / Quantization
AnyBCQ: Hardware Efficient Flexible Binary-Coded Quantization for Multi-Precision LLMs
Gunho Park, Jeongin Bae, Beomseok Kwon, Byeongwook Kim, Se Jung Kwon, Dongsoo Lee
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use fixed-precision optimization instead of progressive multi-precision at 3 bits (ablation) | AnyBCQ · Wikitext perplexity (3-bit, Llama-3.1-8B) | perplexity: 8.08 -> 7.68 | Hurt |
| use fixed-precision optimization instead of progressive multi-precision at 4 bits (ablation) | AnyBCQ · Wikitext perplexity (4-bit, Llama-3.1-8B) | perplexity: 6.84 -> 6.62 | Hurt |
| use fixed-precision optimization instead of multi-precision at 2 bits (ablation) | AnyBCQ · Wikitext perplexity (2-bit, Llama-3.1-8B) | perplexity: 19.01 -> 18.95 | No clear effect |
| share binary bit-planes across precisions instead of separate models | AnyBCQ · Llama-3.1-8B memory footprint | memory (GB): 9.85 -> 4.99 | Improved |
| replace Any-Precision LLM with AnyBCQ at 2 bits | Llama-3.1-8B · MMLU 5-shot (2-bit) | accuracy: 24.66 -> 35.32 | Improved |
| replace Any-Precision LLM with AnyBCQ at 2 bits | Llama-3.1-8B · Wikitext perplexity (2-bit) | perplexity: 1680.77 -> 19.01 | Improved |
| replace Any-Precision LLM with AnyBCQ at 4 bits | Llama-3.1-8B · MMLU 5-shot (4-bit) | accuracy: 64.04 -> 63.15 | Hurt |
| replace Any-Precision LLM with AnyBCQ at 4 bits | Llama-3.1-8B · Wikitext perplexity (4-bit) | perplexity: 6.70 -> 6.84 | Hurt |
| replace Any-Precision LLM kernel with AnyBCQ kernel (2-bit) | Llama-3.1-8B · decoding throughput A100 | tokens/s: 228 -> 245 | Improved |
| replace Any-Precision LLM kernel with AnyBCQ kernel (N=28672, K=8192, 2-bit) | GEMV kernel · GEMV latency | latency (us): 830 -> 747 | Improved |
| replace Any-Precision LLM with AnyBCQ in mixed-precision decoding at 3.6 bits | PMPD setting · CNN/DailyMail | ROUGE-L: 0.154 -> 0.178 | Improved |
| replace Any-Precision LLM with AnyBCQ in mixed-precision decoding at 2.23 bits | PMPD setting · CNN/DailyMail | ROUGE-L: 0.097 -> 0.113 | Improved |
Closest papers by meaning
- Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs (ICML 2024 · similarity 0.92)
- ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMs (NeurIPS 2026 · similarity 0.92)
- ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models (AAAI 2025 · similarity 0.91)
- Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deployment (NeurIPS 2025 · similarity 0.90)
- Progressive Mixed-Precision Decoding for Efficient LLM Inference (ICLR 2025 · similarity 0.90)
- OneBit: Towards Extremely Low-bit Large Language Models (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
