Research map / Architectures and efficiency / Quantization
ZipML: Training Linear Models with End-to-End Low Precision, and a Little Bit of Deep Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use double sampling for end-to-end low-precision SGD with 5-6 bits | linear regression and LS-SVM · Synthetic 100 / gisette | final training loss and convergence rate | No clear effect |
| run quantized data on FPGA instead of 32-bit FPGA or Hogwild! | ZipML linear models · Synthetic 100 / gisette | time to converge: 6-7x faster | Helped |
| replace uniform with data-optimal quantization points (ablation) | quantized linear regression · YearPrediction | bits to converge smoothly: 5-bit -> 3-bit | Helped |
| replace uniform with optimal quantization at same bits (ablation) | quantized linear regression · multiple regression datasets | convergence speed: up to 4x faster | Helped |
| optimal instead of uniform quantization to cut bits (ablation) | quantized linear regression · multiple regression datasets | bits at equal speed: save up to 1.7x bits | Helped |
| increase mini-batch size from 16 to 256 (ablation) | full precision SGD · Figure 6 dataset | epochs to converge | Hurt |
| increase mini-batch size from 16 to 256 (ablation) | 3-bit quantized SGD · Figure 6 dataset | convergence speed | Helped |
| use more features in the dataset | quantized linear regression · multiple regression datasets | bits needed | Hurt |
| replace XNOR5 uniform 5-level quantization with Optimal5 (ablation) | CIFAR-10 CNN · CIFAR-10 | test accuracy: >5 points | Helped |
| apply optimal quantization levels (ablation) | CIFAR-10 CNN · CIFAR-10 | training loss | Helped |
Closest papers by meaning
- QBB: Quantization with Binary Bases for LLMs (NeurIPS 2024 · similarity 0.87)
- SqueezeLLM: Dense-and-Sparse Quantization (ICML 2024 · similarity 0.87)
- Effective Interplay between Sparsity and Quantization: From Theory to Practice (ICLR 2025 · similarity 0.87)
- QuEST: Stable Training of LLMs with 1-Bit Weights and Activations (ICML 2025 · similarity 0.87)
- SLoPe: Double-Pruned Sparse Plus Lazy Low-Rank Adapter Pretraining of LLMs (ICLR 2025 · similarity 0.86)
- DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.