Research map / Image generation / Autoregressive and latent generation
Progressive Supernet Training for Efficient Visual Autoregressive Modeling
Xiaoyue Chen, Yuling Shi, Kaiyuan Li, Huandong Wang, Yong Li, Xiaodong Gu, Xinlei Chen, Mingbao Lin
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use full-depth network on early scales and shallow subnet only on late scales (r7-r10) (ablation) | pretrained VAR-d30, training-free · ImageNet 256x256 | FID: early subnet 12.91 -> late subnet 5.42 | Improved |
| jointly train subnets with shared weights (supernet training) instead of training-free subnet (ablation) | VARiant d=2 subnet · ImageNet 256x256 | FID: 132 -> 2.97 | Improved |
| jointly train subnets with shared weights instead of training-free subnet (ablation) | VARiant d=16 subnet · ImageNet 256x256 | FID: 5.42 -> 2.05 | Improved |
| joint supernet training with a shallower subnet (ablation) | full network VAR-d30 · ImageNet 256x256 | full-network FID: 1.95 -> 2.14 (d=2); 1.95 -> 1.96 (d=16) | Hurt |
| add bridge zone (full depth on scales r1-r6) so skipped layers still get gradients (ablation) | VARiant d16 · ImageNet 256x256 | subnet FID: 9.44 -> 2.05 | Improved |
| add bridge zone (ablation) | VARiant d16 · ImageNet 256x256 | full-network FID: 3.12 -> 1.96 | Improved |
| replace fixed-ratio subnet sampling with three-phase progressive sampling schedule (ablation) | VARiant supernet training · ImageNet 256x256 | FID (full and subnet jointly): fixed p=0.1: 1.96/2.68; p=1.0: 2.32/2.15; progressive: 1.96/2.05 (full/subnet d16) | Improved |
| train subnet only (sampling ratio 1.0) throughout (ablation) | VARiant supernet training · ImageNet 256x256 | full-network FID: 1.96 -> 2.32 | Hurt |
| increase early-scale count N from 6 to 7 at D=16 (ablation) | VARiant d16 · ImageNet 256x256 | FID: FID improves 0.05 with 14% memory increase | Improved |
| increase N from 7 to 10 at D=16 (ablation) | VARiant d16 · ImageNet 256x256 | FID: FID improvements of 0.02, 0.01, 0.01 | No clear effect |
| reduce subnet depth from 30 to 16 layers (ablation) | VARiant · ImageNet 256x256, batch 64 | FID and KV cache: FID 1.95 -> 2.05; KV cache reduced 40.4% | Mixed |
| use single-model VARiant d4 instead of dual-model CoDe | VAR acceleration · ImageNet 256x256 | FID and memory: FID 2.27 -> 2.28; memory 19943MB -> 19582MB | Mixed |
Closest papers by meaning
- Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction (NeurIPS 2024 · similarity 0.92)
- Visual Implicit Autoregressive Modeling (ICML 2026 · similarity 0.90)
- NSARM: Next-Scale Autoregressive Modeling for Robust Real-World Image Super-Resolution (NeurIPS 2026 · similarity 0.90)
- FlexVAR: Flexible Visual Autoregressive Modeling without Residual Prediction (NeurIPS 2025 · similarity 0.89)
- Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compression (NeurIPS 2025 · similarity 0.89)
- Visual Autoregressive Modeling for Image Super-Resolution (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
