Research map / Architectures and efficiency / Quantization
Position-based Scaled Gradient for Model Quantization and Pruning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use position-based gradient scaling toward quantization grid points | ResNet-18 · ImageNet W4A4 | accuracy: 44.3 -> 63.45 | Helped |
| Use PSGD rather than SGD for extremely low-bit quantization | ResNet-18 · ImageNet W3A8 | accuracy: 0.10 -> 66.36 | Helped |
| Use PSGD rather than SGD for extremely low-bit quantization | ResNet-18 · ImageNet W2A8 | accuracy: 0.10 -> 62.65 | Helped |
| Apply PSGD to Adam | Adam on ResNet-32 · CIFAR-100 W4A32 | accuracy: 55.27 -> 60.35 | Helped |
| Use directional rather than independent scaling (ablation) | PSGD on ResNet-18 · ImageNet W2A8 | accuracy: 54.96 -> 56.60 | Helped |
| Remove weight decay (ablation) | PSGD on ResNet-18 · ImageNet W2A8 | accuracy: 51.53 -> 54.96 | Helped |
| Train for 30 additional PSGD epochs (ablation) | PSGD without weight decay on ResNet-18 · ImageNet W2A8 | accuracy: 54.96 -> 62.65 | Helped |
| Train for 30 additional PSGD epochs (ablation) | PSGD · ImageNet W3A8 | accuracy: 64.71 -> 66.36 | Helped |
| Train for 30 additional PSGD epochs (ablation) | PSGD · ImageNet W3A8 | accuracy: 63.76 -> 65.23 | Helped |
| Apply LAPQ to a PSGD-pretrained model | PSGD-pretrained ResNet-18 · ImageNet W4A4 | accuracy: 66.5%; more than 3.1% and 6.2% points higher | Helped |
| Use PSGD targeting 2-bit weights | ResNet-32 · CIFAR-100 W2 | MSE to quantized weights: 157 -> 67 | Helped |
| Use PSGD targeting 2-bit weights | ResNet-32 · CIFAR-100 W2 | accuracy (low precision): SGD low-precision value not reported in this table | Helped |
Closest papers by meaning
- Learning from Loss Landscape: Generalizable Mixed-Precision Quantization via Adaptive Sharpness-Aware Gradient Aligning (ICML 2025 · similarity 0.86)
- Numerical Pruning for Efficient Autoregressive Models (AAAI 2025 · similarity 0.85)
- SLiM: One-shot Quantization and Sparsity with Low-rank Approximation for LLM Weight Compression (ICML 2025 · similarity 0.85)
- Zero-Shot Quantization via Weight-Space Arithmetic (NeurIPS 2026 · similarity 0.85)
- Targeted Low-rank Refinement: Enhancing Sparse Language Models with Precision (ICML 2025 · similarity 0.85)
- Predicting Quantization Price for Selecting PTQ Configurations Before Deployment (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.