Research map / Optimization / Optimizers
PARQ: Piecewise-Affine Regularized Quantization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PARQ, a QAT method using convex piecewise-affine regularization minimized by an aggregate proximal SGD (AProx) with last-iterate convergence, and evaluates on ResNet (CIFAR-10, ImageNet) and DeiT (ImageNet) at ternary and 1-4 bits.
Problem
Quantization-aware training (QAT) relies on the straight-through estimator, a heuristic with poorly understood convergence; existing regularization-based QAT uses nonconvex regularizers and only averaged-iterate guarantees, which do not ensure quantized last iterates.
Approach
Use a convex piecewise-affine regularizer (PAR) whose proximal map induces clustering to discrete values, minimize it with an aggregate proximal SGD (AProx) that applies the prox scaled by the aggregate step size, so the method asymptotically becomes hard quantization (interpreting STE); the quantization set is set online via LSBQ and the slope is annealed.
Key results
- On 1-bit ResNet-20 (CIFAR-10), PARQ reaches 90.48% vs 89.56% for STE; on ternary ResNet-56 it reaches 92.97% vs full precision 93.08%.
- On 1-bit DeiT-S (ImageNet), PARQ reaches 71.06% vs 70.07% for STE.
- On ImageNet ResNet-50, PARQ gives 66.71% at 1-bit vs 66.17% for STE, with a 4-bit result of 74.52% matching STE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PARQ instead of STE/BinaryConnect | ResNet-20 · CIFAR-10, 1-bit | test accuracy: 89.56 -> 90.48 | Helped |
| Use PARQ instead of BinaryRelax | ResNet-56 · CIFAR-10, ternary | test accuracy: 92.34 -> 92.97 | Helped |
| Use PARQ instead of BinaryRelax | ResNet-50 · ImageNet, 1-bit | test accuracy: 66.14 -> 66.71 | Helped |
| Use PARQ instead of STE/BinaryConnect | DeiT-S · ImageNet, 1-bit | test accuracy: 70.07 -> 71.06 | Helped |
| Use PARQ instead of BinaryRelax | DeiT-Ti · ImageNet, ternary | test accuracy: 62.18 -> 60.99 | Hurt |
| Use PARQ instead of BinaryRelax | DeiT-B · ImageNet, 3-bit | test accuracy: 80.94 -> 80.59 | Hurt |
| Gradually anneal PARQ from piecewise-affine soft quantization to hard quantization | PARQ · low-bit quantization, especially smaller models | training stability and final performance | Helped |
Surprising findings
- A convex regularizer can induce clustering toward discrete values, contrary to the expectation that quantization requires nonconvex regularization.
- Prox-SGD with diminishing step sizes loses its quantization effect because the flat segments of the prox map vanish, motivating the aggregate prox step.
Limitations
- Gains over STE and BinaryRelax are often small (sub-point) and within reported standard deviations in many settings.
- Convergence theory covers convex losses only, while experiments use nonconvex deep networks.
- Results are limited to image classification on CIFAR-10 and ImageNet with CNN and ViT backbones.
Experimental setup
Datasets: CIFAR-10 ImageNet
Benchmarks: CIFAR-10 ImageNet
Metrics: test accuracy
Baselines: STE / BinaryConnect BinaryRelax
Builds on: BinaryConnect ProxConnect BinaryRelax ProxQuant LSBQ
Closest papers by meaning
- FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation (CVPR 2025 · similarity 0.87)
- AffineQuant: Affine Transformation Quantization for Large Language Models (ICLR 2024 · similarity 0.87)
- AIQViT: Architecture-Informed Post-Training Quantization for Vision Transformers (AAAI 2025 · similarity 0.87)
- Instance-Aware Group Quantization for Vision Transformers (CVPR 2024 · similarity 0.87)
- Zero-Shot Quantization via Weight-Space Arithmetic (NeurIPS 2026 · similarity 0.87)
- APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformers (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.