Research map / Architectures and efficiency / Quantization
Tequila: Deadzone-free Ternary Quantization for Large Language Models
Hong Huang, Decheng Wu, Rui Cen, Guanghua Yu, Zonghang Li, Kai Liu, Jianchen Zhu, Peng Chen, Xue Liu, Dapeng Wu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| reactivate dead weights in place as signed minima (Minima Reactivation) (ablation) | AbsMean ternary, LLaMA-3.2-1B · ARC-e | accuracy: +1.2% | Improved |
| replace STE-based minima with differentiable reactivation (lambda*w) (ablation) | Minima Reactivation · ARC-e | accuracy: +1.1% | Improved |
| add mixed gradients (hybrid weight and bias role) (ablation) | Tequila w/o Mixed Gradients · ARC-e | accuracy: +1.9% | Improved |
| add Tequila reactivation to AbsMean | AbsMean ternary QAT, LLaMA-3.2-1B · Average of 6 benchmarks | accuracy: 0.445 -> 0.471 | Improved |
| add Tequila reactivation to AbsMean | AbsMean ternary QAT, LLaMA-3.2-3B · Average of 6 benchmarks | accuracy: 0.510 -> 0.530 | Improved |
| add Tequila reactivation to AbsMean | AbsMean ternary QAT, LLaMA-3.2-3B · GPQA | accuracy: 0.301 -> 0.303 | No clear effect |
| add Tequila reactivation | AbsMean ternary QAT, Qwen3-4B · Average of 5 benchmarks | accuracy: 0.5647 -> 0.5985 | Improved |
| use learnable-scale quantizers (LSQ, SEQ, DLT) instead of static AbsMean (ablation) | ternary QAT, LLaMA-3.2-1B · Average of 6 benchmarks | accuracy: 0.445 -> 0.354-0.370 | Hurt |
| vary reactivation parameter lambda (ablation) | Tequila · Average accuracy | accuracy | No clear effect |
| change quantization granularity from per-tensor to per-channel (ablation) | Tequila, LLaMA-3.2-1B · Average accuracy | accuracy: 0.463 -> 0.471 | Improved |
| train Tequila on 10B tokens vs ternary LLMs trained on 100B tokens | Spectra-3.9B · Average of 5 benchmarks (3B) | accuracy: 0.567 -> 0.576 | Improved |
| add dead-weight bias term at inference | Tequila · inference speed (CPU) | speedup vs BF16: 3.0x speedup, bias overhead < 0.1% | No clear effect |
Later papers that compared with this method
1 later accepted paper reports Tequila as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- HESTIA: A Hessian-Guided Differentiable Quantization-Aware Training Framework for Extremely Low-Bit LLMs (NeurIPS 2026 · better on 2 of 2 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- LoTA-QAF: Lossless Ternary Adaptation for Quantization-Aware Fine-Tuning (NeurIPS 2025 · similarity 0.90)
- Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deployment (NeurIPS 2025 · similarity 0.90)
- Treasures in Discarded Weights for LLM Quantization (AAAI 2025 · similarity 0.90)
- Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization (NeurIPS 2025 · similarity 0.90)
- BiLLM: Pushing the Limit of Post-Training Quantization for LLMs (ICML 2024 · similarity 0.90)
- Norm Tweaking: High-Performance Low-Bit Quantization of Large Language Models (AAAI 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
