Research map / Architectures and efficiency / Quantization

RPFQ-ViT: Rotated Phase-Frame Quantization for Extremely Low-Bit Weights in Vision Transformers

NeurIPS 2026 · Architectures and efficiency › Quantization · approach: Post-training quantization of ViTs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RPFQ-ViT, a rotated phase-frame quantizer that pairs ViT linear-layer channels into complex phase planes for extremely low-bit (2-bit weight) QAT, reporting ImageNet gains over Q-ViT/OFQ/LSQ+KD and on-device size and latency benefits.

Problem

Extremely low-bit (2-bit) weight quantization of Vision Transformers causes severe accuracy degradation because scalar real-valued codebooks poorly preserve projection directions.

Approach

Rewrites each even-dimensional nn.Linear as an exact paired-complex widely-linear operator (half-split pairing), then quantizes the complex weights to a learnable rotated phase codebook with learnable anchors, least-squares scale fitting, and residual phase refinement, keeping the real-valued ViT graph unchanged.

Key results

• RPFQ-ViT-B/16 reaches 79.33% Top-1 / 94.48% Top-5 under W2/A4 (three-seed mean), vs 77.89% FP32 reproduced.
• DeiT-S W2/A8 reaches 77.41% Top-1 and Swin-T W2/A8 reaches 79.30% Top-1, above paper-reported Q-ViT and OFQ 2-bit numbers.
• On-device packed 2-bit models shrink size 5.4-7.1x vs FP32 and give 1.41-1.58x median latency speedups on iOS and Android.

Surprising findings

• RPFQ-ViT-B/16 W2/A4 Top-1 (79.33%) exceeds the reproduced FP32 baseline (77.89%) under the same recipe, which the authors attribute to a possible regularization effect.

Limitations

• Evaluation limited to ImageNet-1K classification.
• Mobile speedups depend on runtime support for packed weights; no energy measurements.
• A2 activation quantization remains unstable; W/A settings vary across backbones (A4 for ViT-B/16, A8 for DeiT-S/Swin-T).

Experimental setup

Datasets: ImageNet-1K

Benchmarks: ImageNet-1K

Metrics: Top-1 accuracy Top-5 accuracy weight MSE angular error attention score MSE feature cosine model size latency

Baselines: Q-ViT OFQ LSQ+KD LSQ+ Real LSQ FP32

Builds on: LSQ Q-ViT OFQ FrameQuant Deep Complex Networks

Closest papers by meaning

• GradQ-ViT: Robust and Efficient Gradient Quantization for Vision Transformers (AAAI 2025 · similarity 0.90)
• GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers (NeurIPS 2025 · similarity 0.90)
• AIQViT: Architecture-Informed Post-Training Quantization for Vision Transformers (AAAI 2025 · similarity 0.89)
• MimiQ: Low-Bit Data-Free Quantization of Vision Transformers with Encouraging Inter-Head Attention Similarity (AAAI 2025 · similarity 0.89)
• Bi-ViT: Pushing the Limit of Vision Transformer Quantization (AAAI 2024 · similarity 0.89)
• APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformers (CVPR 2025 · similarity 0.89)