Research map / Architectures and efficiency / Quantization
Reshape and Adapt for Output Quantization (RAOQ): Quantization-aware Training for In-memory Computing Systems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RAOQ, a quantization-aware training framework (A-shift, W-reshape, BitAug, ADC-LoRA) that mitigates ADC quantization error in in-memory computing so CNNs and transformers keep near-baseline accuracy at low ADC bit precision.
Problem
ADC quantization in analog in-memory computing has fixed quantization step and clipping, which causes large accuracy degradation for modern CNN and transformer models, and vanilla QAT fails to recover it.
Approach
Improve SQNR of ADC inputs by shifting unsigned activations to signed (A-shift) and applying a kurtosis penalty to reshape weight distributions (W-reshape); adapt models by randomly sampling neighboring ADC bit precisions during training (BitAug); and reduce training cost with an MSE-initialized LoRA (ADC-LoRA).
Key results
- On ResNet50/ImageNet with 4-bit weights/activations and 8-bit ADC, RAOQ reaches 76.27% vs 72.05% for vanilla QAT and 76.31% without ADC.
- On BERT-base SQuAD 1.1 with 7-bit ADC, RAOQ achieves 87.75 F1 vs 64.46 for vanilla QAT (full precision 88.58).
- ADC-LoRA gives over 45x reduction in trainable parameters (e.g., BERT-large 345M to 3.5M trainable parameters for the LoRA setting).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply RAOQ instead of vanilla QAT | ResNet50 · ImageNet | accuracy: 72.05 -> 76.27 | Helped |
| Apply RAOQ instead of vanilla QAT | BERT-base · SQuAD 1.1 | F1: 82.43 -> 87.67 | Helped |
| Apply RAOQ instead of vanilla QAT | OPT-1.3B · WikiText-2 | perplexity: 30.14 -> 14.42 | Helped |
| Apply RAOQ instead of vanilla QAT | YOLOv5s · COCO2017 | mAP: 20.32 -> 28.49 | Helped |
| Add A-shift (ablation) | ResNet50 · ImageNet | accuracy: 72.05 -> 75.77 | Helped |
| Add W-reshape (ablation) | ResNet50 · ImageNet | accuracy: 72.05 -> 75.01 | Helped |
| Add BitAug (ablation) | ResNet50 · ImageNet | accuracy: 72.05 -> 75.65 | Helped |
| Add A-shift and W-reshape (ablation) | ResNet50 · ImageNet | accuracy: 72.05 -> 76.02 | Helped |
| Add A-shift, W-reshape, and BitAug (ablation) | ResNet50 · ImageNet | accuracy: 72.05 -> 76.27 | Helped |
| Add ADC-LoRA (ablation) | BERT-large · SQuAD 1.1 | F1: 89.08 -> 89.28 | Helped |
| Add ADC-LoRA (ablation) | OPT-1.3B · WikiText-2 | perplexity: 14.39 -> 14.42 | Hurt |
| Add ADC-LoRA (ablation) | BLOOM-1.7B · WikiText-2 | perplexity: 17.05 -> 17.06 | Hurt |
Surprising findings
- Vanilla QAT with ADC quantization can collapse perplexity badly (e.g., OPT-1.3B at 7-bit ADC reaches 465-ish to 600+ perplexity range), showing ADC quantization is far more damaging than standard quantization.
Limitations
- Analog noise is only modeled with simple additive noise and is left for future study.
- BitAug with all ADC bits simultaneously is computationally expensive; sampling is used instead.
- Evaluated on simulated ADC quantization rather than fabricated IMC hardware.
Experimental setup
Datasets: ImageNet COCO 2017 SQuAD 1.1 WikiText-2 WikiText-103 CIFAR-10
Benchmarks: ImageNet COCO 2017 SQuAD 1.1 WikiText-2 WikiText-103 CIFAR-10
Metrics: top-1 accuracy mAP F1 perplexity SQNR
Baselines: vanilla QAT Jin et al. 2022 Sun et al. 2021 Wei et al. 2020
Builds on: LSQ+ LoRA NetAug kurtosis regularization for quantization (Shkolnik et al., 2020)
Closest papers by meaning
- Analog Foundation Models (NeurIPS 2025 · similarity 0.87)
- A2Q+: Improving Accumulator-Aware Weight Quantization (ICML 2024 · similarity 0.85)
- AIQViT: Architecture-Informed Post-Training Quantization for Vision Transformers (AAAI 2025 · similarity 0.85)
- JAQ: Joint Efficient Architecture Design and Low-Bit Quantization with Hardware-Software Co-Exploration (AAAI 2025 · similarity 0.85)
- Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation (AAAI 2024 · similarity 0.85)
- Towards Exact Gradient-based Training on Analog In-memory Computing (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.