Research map / Architectures and efficiency / Quantization
VAR-Q: Tuning-free KV Cache Quantization for Visual Autoregressive Image and Video Generation
Boxun Xu, Jiaji Lu, Yuxuan Yin, Zihu Wang, Ziyue (Alvin) Liu, Yu Wang, Zirui Liu, Peng Li
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VAR-Q, a tuning-free KV cache quantization method for visual autoregressive image and video generation, motivated by KV caches whose statistics differ from language models.
Problem
KV memory in visual autoregressive generation (next-scale prediction for images/video and next-frame prediction for video) exceeds model weight memory at scale, limiting deployment because of the large number of visual tokens.
Approach
Analyzes KV caches of block-wise visual autoregressive decoding, finds their statistics diverge from language-model KV caches, and uses this analysis to design a quantization scheme that requires no tuning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| group KV quantization per (channel, block) instead of per-channel only (ablation) | VAR-Q · Infinity-2B INT4 | attention output error eps_O (%): 13.13 -> 10.13 | Improved |
| add block-axis grouping to channel grouping (ablation) | VAR-Q · VAR-d30 INT4 | FID: 2.83 -> 1.97 | Improved |
| add channel-axis grouping (ablation) | VAR-Q · VAR-d30 INT4 | FID: 12.07 -> 1.97 | Improved |
| add channel-axis grouping (ablation) | VAR-Q · Self-Forcing INT4 | VBench Total Quality: 67.22 -> 80.99 | Improved |
| use block-aware grouping instead of per-token & per-head (ablation) | VAR-Q · VAR-d30 INT4 | FID: 2.21 -> 1.97 | Improved |
| align group boundaries with block boundaries (ablation) | VAR-Q · Self-Forcing INT4 | VBench Total Quality: 80.46 -> 80.91 | Improved |
| use finer sub-block ratio r=1/4 instead of r=1 (ablation) | VAR-Q · Self-Forcing INT4 | VBench Total Quality: 80.99 -> 81.34 | Improved |
| zero top 0.5% outlier channels vs equal-sized inlier set (ablation) | LongLive-1.3B KV cache · LongLive-1.3B | CLIP score: 0.32 -> 0.28 (outlier); 0.32 (inlier) | Hurt |
| use VAR-Q instead of KIVI at INT3 on InfinityStar 720p | KIVI · InfinityStar-8B 720p INT3 | VBench Total Quality: 75.46 -> 80.59 | Improved |
| use VAR-Q instead of FlexGen at INT3 on VAR-d20 | FlexGen · VAR-d20 INT3 | FID: 15.33 -> 3.77 | Improved |
| quantize at INT4 vs INT3 with Infinity-8B | VAR-Q · Infinity-8B | GenEval Overall: 83.28 (INT3) -> 74.38 (INT2) | Hurt |
| VAR-Q at INT4 on VAR-d20 vs BF16 | VAR-Q · VAR-d20 | FID: 4.11 -> 3.44 | No clear effect |
Closest papers by meaning
- Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compression (NeurIPS 2025 · similarity 0.92)
- HeatKV: Head-tuned KV-cache Compression for Visual Autoregressive Modeling (NeurIPS 2026 · similarity 0.90)
- VideoMLA: Low-Rank Latent KV Cache for Minute-Scale Autoregressive Video Diffusion (NeurIPS 2026 · similarity 0.89)
- Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction (NeurIPS 2024 · similarity 0.89)
- VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Acceleration (ICLR 2025 · similarity 0.88)
- Collaborative Decoding Makes Visual Auto-Regressive Modeling Efficient (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
