Research map / Image generation / Autoregressive and latent generation
HMAR: Efficient Hierarchical Masked Auto-Regressive Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HMAR, a Markovian next-scale autoregressive image generator with intra-scale multi-step masked refinement and block-sparse Triton attention kernels, improving speed, memory and quality over VAR on ImageNet.
Problem
VAR's next-scale autoregressive image generation suffers from conditionally independent token sampling within scales, superlinearly growing sequence lengths with resolution, and inability to change sampling steps without retraining.
Approach
Reformulate next-scale prediction as a Markov process conditioned only on the reconstruction of the previous scale (enabling block-diagonal windowed attention and no KV cache), and add MaskGIT-style multi-step masked refinement within each scale, trained with log-normal per-scale loss reweighting.
Key results
- HMAR-d30 reaches FID 1.95 and IS 334.5 on ImageNet 256x256 with 14 steps, versus VAR-d30 at FID 1.95 and IS 303.6.
- HMAR yields up to 2.5x faster end-to-end training and 1.75x faster inference than VAR, with over 3x lower inference memory footprint.
- On ImageNet 512x512, HMAR-d24 (1.3B) achieves FID 2.99 and IS 304.1, versus VAR-d36 (2.5B) at FID 2.63 and IS 303.2, using about half the parameters.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace VAR-d16 with HMAR-d16 | VAR-d16 · ImageNet 256×256 | FID: 3.50 -> 3.01 | Helped |
| replace VAR-d16 with HMAR-d16 | VAR-d16 · ImageNet 256×256 | IS: 276.0 -> 288.6 | Helped |
| use HMAR-d24 instead of VAR-d36 | VAR-d36 · ImageNet 512×512 | IS: 303.2 -> 304.1 | Helped |
| use HMAR-d24 instead of VAR-d36 | VAR-d36 · ImageNet 512×512 | FID: 2.63 -> 2.99 | Hurt |
| use Markovian next-scale conditioning (ablation) | HMAR-d16 · ImageNet 256×256 | FID: 3.50 -> 3.76 | Hurt |
| use Markovian next-scale conditioning (ablation) | HMAR-d16 · ImageNet 256×256 | IS: 276.0 -> 293.3 | Helped |
| add loss weighting (ablation) | HMAR-d16 with Markov assumption · ImageNet 256×256 | FID: 3.76 -> 3.42 | Helped |
| add loss weighting (ablation) | HMAR-d16 with Markov assumption · ImageNet 256×256 | IS: 293.3 -> 307.9 | Helped |
| add masked prediction (ablation) | HMAR-d16 with loss weighting · ImageNet 256×256 | FID: 3.42 -> 3.01 | Helped |
| add masked prediction (ablation) | HMAR-d16 with loss weighting · ImageNet 256×256 | IS: 307.9 -> 288.6 | Hurt |
| scale up from HMAR-d16 to HMAR-d30 (ablation) | HMAR · ImageNet 256×256 | FID: 3.01 -> 1.95 | Helped |
| scale up from HMAR-d16 to HMAR-d30 (ablation) | HMAR · ImageNet 256×256 | IS: 288.6 -> 334.5 | Helped |
Surprising findings
- On ImageNet 512x512, HMAR-d24 has a worse FID (2.99) than VAR-d36 (2.63) despite its higher IS, so it is not uniformly better.
Limitations
- Only class-conditional ImageNet generation evaluated; text-to-image left to future work.
- 512x512 model uses about 2x fewer parameters than VAR-d36 and is not uniformly better in FID.
- Relies on VAR's pretrained multi-scale VQ-VAE tokenizer; tokenizer improvements left for future work.
Experimental setup
Datasets: ImageNet
Benchmarks: ImageNet 256x256 ImageNet 512x512
Metrics: FID Inception Score Precision Recall training runtime inference runtime memory footprint
Baselines: VAR-d16 VAR-d20 VAR-d24 VAR-d30 VAR-d36 DiT-XL/2 MaskGIT MAR-L MAGE VQGAN LlamaGen HART
Builds on: VAR MaskGIT FlashAttention Triton
Closest papers by meaning
- Hierarchical Masked Autoregressive Models with Low-Resolution Token Pivots (ICML 2025 · similarity 0.91)
- Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction (NeurIPS 2024 · similarity 0.91)
- ControlAR: Controllable Image Generation with Autoregressive Models (ICLR 2025 · similarity 0.90)
- RandAR: Decoder-only Autoregressive Visual Generation in Random Orders (CVPR 2025 · similarity 0.89)
- EditAR: Unified Conditional Generation with Autoregressive Models (CVPR 2025 · similarity 0.89)
- NSARM: Next-Scale Autoregressive Modeling for Robust Real-World Image Super-Resolution (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.