Research map / Image generation / Autoregressive and latent generation
Halton Scheduler for Masked Generative Image Transformer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a Halton-sequence-based token unmasking scheduler for MaskGIT that spreads tokens spatially, improving FID and diversity over the Confidence scheduler without retraining.
Problem
MaskGIT's token unmasking scheduler (Confidence) selects clustered, highly certain tokens, causing high mutual information among tokens unmasked in the same step and low information gain, which hurts image quality and diversity.
Approach
Replace the confidence-based unmasking order with a fixed, deterministic order derived from a 2D Halton low-discrepancy sequence (bases 2 and 3), spreading tokens uniformly across the grid at each step; motivated by a mutual-information decomposition of the sampling objective.
Key results
- On ImageNet 256x256 with the 705M model at 32 steps, Halton reaches FID 3.74 vs 5.93 for Confidence.
- On ImageNet 512x512 with the public 246M MaskGIT reproduction at 32 steps, Halton reaches FID 6.11 vs 8.38 for Confidence.
- On zero-shot COCO text-to-image, Halton reaches FID 28.8 vs 31.5 for Confidence and 38.9 for aMused.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the Halton scheduler instead of the Confidence scheduler (ablation) | Our MaskGIT · ImageNet 256×256, 32 steps, 484M | FID: 7.5 -> 5.3 | Helped |
| Use the Halton scheduler instead of the Random scheduler (ablation) | Our MaskGIT · ImageNet 256×256, 32 steps, 484M | FID: 9.7 -> 5.3 | Helped |
| Use the Halton scheduler instead of the Confidence scheduler (ablation) | Our MaskGIT · ImageNet 256×256, 12 steps, 705M | FID: 6.80 -> 6.19 | Helped |
| Use the Halton scheduler instead of the Confidence scheduler (ablation) | Our MaskGIT · ImageNet 256×256, 32 steps, 705M | FID: 5.93 -> 3.74 | Helped |
| Use the Halton scheduler instead of the Confidence scheduler (ablation) | MaskGIT · ImageNet 512×512, 12 steps | FID: 7.60 -> 7.15 | Helped |
| Use the Halton scheduler instead of the Confidence scheduler (ablation) | MaskGIT · ImageNet 512×512, 32 steps | FID: 8.38 -> 6.11 | Helped |
| Increase Halton scheduler steps from 24/32 to 32/32 (ablation) | MaskGIT · ImageNet 512×512 intermediate generation | FID: 6.25 -> 6.11 | Helped |
| Increase Halton scheduler steps from 24/32 to 32/32 (ablation) | MaskGIT · ImageNet 512×512 intermediate generation | IS: 174.3 -> 184.0 | Helped |
| Use the Halton scheduler instead of the Confidence scheduler (ablation) | Our MaskGIT · COCO zero-shot text-to-image | FID: 31.5 -> 28.8 | Helped |
| Use the Halton scheduler instead of the Random scheduler (ablation) | Our MaskGIT · COCO zero-shot text-to-image | FID: 33.3 -> 28.8 | Helped |
Surprising findings
- The Confidence scheduler gets worse as the number of sampling steps increases, while the Halton scheduler keeps improving.
- A purely random scheduler performs much worse than both Confidence and Halton at 512x512 (FID 18.84 at 12 steps).
Limitations
- Failure modes include duplicated objects (multiple heads or instances) because initial tokens spread across the image.
- Long-range dependencies are not handled; MIM cannot self-correct earlier predictions.
- Assumes the trained model accurately estimates token marginals; miscalibration is not accounted for.
Experimental setup
Datasets: ImageNet CC12M Segment Anything (SA-1B subset) JourneyDB DiffusionDB COCO
Benchmarks: ImageNet 256x256 ImageNet 512x512 COCO zero-shot
Metrics: FID IS Precision Recall CLIP score
Baselines: MaskGIT Confidence scheduler Random scheduler aMused MaskGIT Token-Critics AutoNAT-L FSQ MAGE DiT-XL/2 StyleGAN-XL LlamaGen-3B LDM 2.1
Builds on: MaskGIT Token-Critics Halton sequence MUSE
Closest papers by meaning
- Masked Generative Nested Transformers with Decode Time Scaling (ICML 2025 · similarity 0.88)
- Fixed-Point Masked Generative Modeling (NeurIPS 2026 · similarity 0.85)
- Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesis (ICLR 2025 · similarity 0.84)
- ENAT: Rethinking Spatial-temporal Interactions in Token-based Image Synthesis (NeurIPS 2024 · similarity 0.84)
- Unlocking the Capabilities of Masked Generative Models for Image Synthesis via Self-Guidance (NeurIPS 2024 · similarity 0.84)
- Plug-and-Play Context Feature Reuse for Efficient Masked Generation (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.