Research map / Architectures and efficiency / Quantization
AQ-DETR: Low-Bit Quantized Detection Transformer with Auxiliary Queries
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AQ-DETR, a 4-bit quantization-aware training method for DETR and Deformable DETR that adds auxiliary queries for extra one-to-many matching during training and layer-by-layer distillation of encoder attention, achieving near full-precision AP on VOC and COCO.
Problem
Full low-bit (weight-activation-attention) quantization of DETR-based detectors causes severe accuracy drops because quantizing object queries and encoder features reduces query-to-positive-sample matching quality.
Approach
Add an auxiliary matching branch with extra auxiliary queries (matched to repeated ground truth) during training only to increase matched queries and enhance query capacity, plus a layer-by-layer distillation that aligns quantized encoder attention with full-precision attention, with no test-time cost.
Key results
- On COCO, 4-4-4 bit AQ-DETR with Deformable DETR-R50 reaches 44.1 AP vs 47.0 full-precision and 35.2 for LSQ+.
- On COCO, 4-4-4 bit AQ-DETR with DETR-R50 reaches 40.2 AP vs 42.0 full-precision and 37.4 for Q-DETR.
- Ablation on Deformable DETR: AQ adds 5.8 AP and LLD adds 3.3 AP over LSQ+; combined they give 44.1 AP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AQ-DETR instead of Q-DETR | DETR-R50, 4-4-4-bit quantization · VOC test2007 | AP: 50.4 -> 53.7 | Helped |
| Use AQ-DETR instead of Q-DETR | Deformable DETR-R50, 4-4-4-bit quantization · VOC test2007 | AP: 61.1 -> 63.1 | Helped |
| Use AQ-DETR instead of Q-DETR | DETR-R50, 4-4-4-bit quantization · COCO val2017 | AP: 37.4 -> 40.2 | Helped |
| Use AQ-DETR instead of Q-DETR | Deformable DETR-R50, 4-4-4-bit quantization · COCO val2017 | AP: 40.7 -> 44.1 | Helped |
| Add auxiliary queries (ablation) | Deformable DETR-R50 with LSQ+ · COCO | AP: 35.2 -> 41.0 | Helped |
| Add layer-by-layer distillation (ablation) | Deformable DETR-R50 with LSQ+ · COCO | AP: 35.2 -> 38.5 | Helped |
| Add auxiliary queries and layer-by-layer distillation (ablation) | Deformable DETR-R50 with LSQ+ · COCO | AP: 35.2 -> 44.1 | Helped |
| Increase auxiliary query count from 300 to 1500 (ablation) | AQ-DETR · COCO | AP: 38.5 -> 44.1 | Helped |
| Increase auxiliary query count from 1500 to 1800 (ablation) | AQ-DETR · COCO | AP: 44.1 -> 43.8 | Hurt |
| Replace cross-entropy distillation loss with KL divergence (ablation) | AQ-DETR · COCO | AP: 44.1 -> 43.3 | Hurt |
| Replace cross-entropy distillation loss with cosine similarity (ablation) | AQ-DETR · COCO | AP: 44.1 -> 39.7 | Hurt |
| Remove distillation (ablation) | AQ-DETR · COCO | AP: 44.1 -> 41.0 | Hurt |
Surprising findings
- Cosine similarity as the distillation loss reduces AP (39.7) relative to no distillation's ancestor baseline, while cross-entropy gives the best result (44.1).
- Distilling from a pre-trained full-precision encoder as teacher performed worse (42.4) than the proposed layer-by-layer distillation (44.1).
Limitations
- Evaluated only on DETR and Deformable DETR, not other DETR variants (authors note future work).
- Remaining gap to full precision on COCO (e.g., 3.6% AP for Deformable DETR, 1.8 AP for DETR), and training requires extra auxiliary queries and matching cost.
Experimental setup
Datasets: PASCAL VOC COCO 2017 ImageNet ILSVRC12
Benchmarks: PASCAL VOC test2007 COCO val2017
Metrics: AP model size (MB) OPs (G)
Baselines: LSQ LSQ+ Q-DETR Percentile VT-PTQ
Builds on: LSQ+ DETR Deformable DETR Q-DETR H-DETR
Closest papers by meaning
- Salience DETR: Enhancing Detection Transformer with Hierarchical Salience Filtering Refinement (CVPR 2024 · similarity 0.88)
- SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration (ICLR 2025 · similarity 0.86)
- KD-DETR: Knowledge Distillation for Detection Transformer with Consistent Distillation Points Sampling (CVPR 2024 · similarity 0.86)
- ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantization (NeurIPS 2025 · similarity 0.86)
- Reg-PTQ: Regression-specialized Post-training Quantization for Fully Quantized Object Detector (CVPR 2024 · similarity 0.86)
- DETRs Beat YOLOs on Real-time Object Detection (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.