Research map / Image generation / Autoregressive and latent generation
Plug-and-Play Context Feature Reuse for Efficient Masked Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ReCAP, a training-free plug-and-play module that speeds up masked generative models by interleaving full evaluations with cheap local steps that reuse cached KV features of unchanged context tokens.
Problem
Masked generative models need many iterative decoding steps for high-fidelity image synthesis, and each bidirectional step recomputes attention over all tokens, so inference is costly; reducing steps degrades fidelity because parallel decoding ignores inter-token dependencies.
Approach
Observes that cosine similarity of context token features stays near 1 when few tokens are newly unmasked, so grouped decoding performs one full evaluation per group and then several local steps that recompute QKV only for the newly decoded tokens while reusing cached KVs for the rest.
Key results
- MAR-H+ReCAP with (96+32) steps reaches FID 1.57 at 1.0s per image vs. MAR-H 256 steps FID 1.56 at 2.4s, about 2.4x faster.
- MaskGIT-r+ReCAP at 32 total steps reduces time from 0.189s to 0.137s per image with FID 3.97 vs 3.98.
- MAR-L+ReCAP (64+20) achieves FID 1.80 in 0.33s, outperforming MAR-L at 64 steps (FID 1.83, 0.31s).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace 10 Full-FEs with Local-FEs | MaskGIT-r · ImageNet256 class-conditional, no CFG, 32 steps | Time per image: 0.19 -> 0.14 | Helped |
| Replace 10 Full-FEs with Local-FEs | MaskGIT-r · ImageNet256 class-conditional, no CFG, 32 steps | FID: 3.97 -> 3.98 | No clear effect |
| Reuse context features in 32 Local-FEs | MAR-Huge · ImageNet256 class-conditional, with CFG | Time: 2.40 -> 1.00 | Helped |
| Reuse context features in 32 Local-FEs | MAR-Huge · ImageNet256 class-conditional, with CFG | FID: 1.56 -> 1.57 | No clear effect |
| Reuse context features in 20 Local-FEs | MAR-Large · ImageNet256 class-conditional, with CFG | FID: 1.83 -> 1.80 | Helped |
| Reuse context features in 20 Local-FEs | MAR-Large · ImageNet256 class-conditional, with CFG | Time: 0.31 -> 0.33 | Mixed |
| Reuse context features in Local-FEs | MAGE · ImageNet 256×256 unconditional, 128 NFE | Time: 1.56 -> 1.018 | Helped |
| Reuse context features in Local-FEs | MAGE · ImageNet 256×256 unconditional, 128 NFE | FID: 7.12 -> 7.14 | No clear effect |
| Use 8 Local-FEs instead of Full-FEs (u=0) (ablation) | MaskGIT-r · ImageNet256 class-conditional, no CFG, 16 steps | FID: 4.46 -> 5.02 | Hurt |
| Use 4 Local-FEs and 12 Full-FEs (u=8) (ablation) | MaskGIT-r · ImageNet256 class-conditional, no CFG, 16 steps | FID: 4.46 -> 4.50 | No clear effect |
| Use 5 Local-FEs and 15 Full-FEs (u=10) (ablation) | MaskGIT-r · ImageNet256 class-conditional, no CFG, 20 steps | FID: 4.18 -> 4.23 | No clear effect |
| Use 8 Local-FEs and 16 Full-FEs (u=10) (ablation) | MaskGIT-r · ImageNet256 class-conditional, no CFG, 24 steps | FID: 4.09 -> 4.09 | No clear effect |
Surprising findings
- Reducing MAR-L to 20 steps degrades FID to 3.12, while ReCAP with (20+8) steps reaches 2.41 at similar cost, showing local steps help at low step counts.
Limitations
- Effectiveness depends on the base model improving with more decoding steps.
- Assumption of stable context features holds best in high-step regimes, so benefit is smaller for small models or short schedules.
- Adaptive or learned insertion of local steps is left for future work.
Experimental setup
Datasets: ImageNet256
Benchmarks: ImageNet256
Metrics: FID IS inference time per image
Baselines: MaskGIT MAR MAGE Token-Critic DPC StraIT AutoNAT U-ViT DiT REPA VAR
Builds on: MaskGIT MAR MAGE KV caching
Closest papers by meaning
- Fixed-Point Masked Generative Modeling (NeurIPS 2026 · similarity 0.90)
- MaskSense: Confronting the Visual Exploration Trap in Masked Image Generation (NeurIPS 2026 · similarity 0.88)
- When Worse is Better: Navigating the Compression Generation Trade-off In Visual Tokenization (NeurIPS 2025 · similarity 0.87)
- Efficient Generative Modeling with Residual Vector Quantization-Based Tokens (ICML 2025 · similarity 0.87)
- BiGR: Harnessing Binary Latent Codes for Image Generation and Improved Visual Representation Capabilities (ICLR 2025 · similarity 0.87)
- Collaborative Decoding Makes Visual Auto-Regressive Modeling Efficient (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.