Research map / Architectures and efficiency / Quantization
ReQAT: Achieving Full-Precision Reasoning Accuracy with 4-bit Floating-Point Quantization-Aware Training
Janghwan Lee, Sihwa Lee, Jinseok Kim, Yongjik Kim, Jieun Lim, Jinwook Oh, Jungwook Choi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add trace alignment to two-stage QAT (ablation) | FT+QAT · MXFP4 W4A16 R1-Qwen-14B (350M) | AIME accuracy: 62.60 -> 67.29 | Improved |
| add Q-FIT (pre-RoPE scaling + post-RoPE shifting) to TAQ (ablation) | ReQAT_T · NVFP4 W4A4KV4 R1-Qwen-14B (280M) | AIME accuracy: 63.13 -> 65.94 | Improved |
| use only one of scale or shift in Q-FIT (ablation) | ReQAT_TQ · NVFP4 W4A4KV4 R1-Qwen-14B | AIME accuracy: 65.94 -> 62.71 | Hurt |
| add SEM to TAQ+Q-FIT (ablation) | ReQAT_TQ · R1-Llama-8B AIME-120 NVFP4 W4A4KV4 | AIME accuracy: 40.32 -> 41.85 | Improved |
| add SEM to TAQ+Q-FIT (ablation) | ReQAT_TQ · R1-Llama-8B GSM8K NVFP4 W4A4KV4 | accuracy: 89.86 -> 89.85 | No clear effect |
| add SEM to TAQ+Q-FIT (ablation) | ReQAT_TQ · R1-Llama-8B MATH-500 NVFP4 W4A4KV4 | accuracy: 90.72 -> 90.53 | No clear effect |
| use soft weighting instead of binary mask in SEM (ablation) | SEM · MXFP4 W4A4 AIME-90 (210M) | AIME accuracy: 63.19 -> 65.14 | Improved |
| train more epochs of TAQ instead of adding SEM (ablation) | multi-epoch TAQ · R1-Llama-8B AIME NVFP4 | AIME accuracy: 38.34 -> 37.71 | No clear effect |
| add TAQ over standard QAT | ReQAT_T vs QAT · NVFP4 W4A4KV4 R1-Qwen-14B (280M) | AIME accuracy: 58.86 -> 63.13 | Improved |
| add more BF16 FT epochs before PTQ (ablation) | FT then PTQ · R1-Llama-8B NVFP4 W4A4KV4 | AIME accuracy: 11.05 -> 15.42 | Hurt |
| add QAT after fine-tuning without trace alignment (ablation) | FT+QAT · MXFP4 W4A16 R1-Qwen-14B (350M) | AIME accuracy: 62.29 -> 62.60 | No clear effect |
| quantize with ReQAT vs BF16 FT | ReQAT · NVFP4 W4A4KV4 R1-Qwen-14B | AIME accuracy: 65.46 -> 65.94 | Improved |
Closest papers by meaning
- ReSET: Accurate Latency-Critical NVFP4 Reasoning via Step-Aware Temperature Scaling (NeurIPS 2026 · similarity 0.91)
- LAQuant: A Simple Overhead-free Large Reasoning Model Quantization by Layer-wise Lookahead Loss (NeurIPS 2026 · similarity 0.91)
- How Far Are We from Optimal Reasoning Efficiency? (NeurIPS 2025 · similarity 0.87)
- When Reasoning Meets Compression: Understanding the Effects of LLMs Compression on Large Reasoning Models (ICLR 2026 · similarity 0.87)
- VeriThinker: Learning to Verify Makes Reasoning Model Efficient (NeurIPS 2025 · similarity 0.87)
- SmartThinker: Progressive Chain-of-Thought Length Calibration for Efficient Large Language Model Reasoning (ICML 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
