Research map / Architectures and efficiency / Quantization
CoCoQuant: Breaking the Bandwidth Wall via Co-Optimized Communication and Computation Quantization
Haojie Duanmu, Jifeng Ding, Size Zheng, Xuegui Zheng, Jiangfei Duan, Xingcheng Zhang, Li-Wen Chang, Xin Liu, Dahua Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| quantize only computation (W4A4C16) | Llama3.3-70B-Instruct · 8x RTX 4090 | speedup: 1.31x speedup with PPL 3.63 -> 7.30 | Hurt |
| add communication quantization with mixed precision (CoCoQuant 2) instead of W8A8C8 | Llama3.3-70B-Instruct · 8x RTX 4090 | speedup: 1.82x -> 2.09x | Improved |
| use CoCoQuant 1 instead of uniform W4A4C4* | Llama3-8B-Instruct · WikiText-2 | perplexity: 9.21 -> 7.87 | Improved |
| use CoCoQuant 1 instead of uniform W4A4C4* | Qwen3-MoE-235B · 8x RTX 4090 | speedup: 2.01x -> 2.92x | Improved |
| apply W4A4C16 to MoE | Llama4-Scout-109B · WikiText-2 | perplexity: 7.76 -> 457.96 | Hurt |
| add precision-aligned graph rewriting (PGR) (ablation) | CoCoQuant W4A4C4 · Llama3.3-70B WikiText-2 | perplexity: 6.29 -> 4.87 | Improved |
| add precision-aligned graph rewriting (PGR) (ablation) | CoCoQuant W4A4C4 · Llama3.3-70B | speedup: 2.78x -> 3.52x | Improved |
| add precision-aligned graph rewriting (PGR) (ablation) | CoCoQuant W8A8C8 · Llama3.3-70B WikiText-2 | perplexity: 3.76 -> 3.66 | Improved |
| add precision-aligned graph rewriting (PGR) (ablation) | CoCoQuant W8A8C8 · Llama3.3-70B | speedup: 1.74x -> 2.05x | Improved |
| use ILP mixed precision instead of uniform at W4A4C4 latency (ablation) | CoCoQuant · GSM8K Llama3-8B-Instruct | accuracy (%): 46.63 -> 73.62 | Improved |
| use ILP mixed precision instead of uniform at W8A8C8 latency (ablation) | CoCoQuant · GSM8K Llama3-8B-Instruct | accuracy (%): 76.12 -> 76.22 | No clear effect |
| use ILP mixed precision instead of uniform at W4A4C4 latency (ablation) | CoCoQuant · GSM8K Llama3-8B | accuracy (%): 21.23 -> 43.82 | Improved |
Closest papers by meaning
- ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMs (NeurIPS 2026 · similarity 0.88)
- SDP4Bit: Toward 4-bit Communication Quantization in Sharded Data Parallelism for LLM Training (NeurIPS 2024 · similarity 0.88)
- LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Grid (ICLR 2025 · similarity 0.87)
- KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization (NeurIPS 2024 · similarity 0.87)
- Cherry on Top: Parameter Heterogeneity and Quantization in Large Language Models (NeurIPS 2024 · similarity 0.87)
- KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
