Research map / Image generation / Autoregressive and latent generation

BitDance: Scaling Autoregressive Generative Models with Binary Tokens

NeurIPS 2026 · Image generation › Autoregressive and latent generation · approach: Next-scale and token autoregressive images · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes BitDance, an autoregressive image generator over binary visual tokens (up to 2^256 vocabulary) with a binary diffusion head and next-patch diffusion for parallel prediction, reaching FID 1.24 on ImageNet and GenEval 0.86 at 14B.

Problem

Autoregressive visual generation struggles with token design (VQ reconstruction loss, continuous-VAE error accumulation) and with sampling from very large discrete vocabularies, while parallel AR decoding ignores joint dependencies among simultaneously sampled tokens.

Approach

Scale binary (LFQ-style) token entropy up to 2^256 with group-wise entropy loss, sample each binary token vector with a rectified-flow diffusion head embedded in continuous space and hard-binarized at the end, and extend it to next-patch prediction with block-wise causal attention so that patches of tokens are sampled jointly.

Key results

• BitDance-H-1x (1B) reaches FID 1.24 on ImageNet 256x256 class-conditional generation, the best among listed AR models.
• BitDance-B-4x (260M) reports FID 1.69 vs. 2.15 for the 1.4B RandAR-XXL, at 8.7x speedup per the intro.
• The 14B text-to-image model scores 0.86 on GenEval, 88.28 on DPG-Bench, 0.532 on OneIG-EN and 0.512 on OneIG-ZH, and 12.4 s latency at 1024px versus 402 s for NextStep-1.

Surprising findings

• A bit-wise classification head reaches only FID 8.37 versus 1.79 for the binary diffusion head, and a full token classification head runs out of memory.

Limitations

• Text-to-image training data (under 450M pairs) and the proprietary SFT images from Seedream and Z-Image-Turbo limit reproducibility.
• Weaker on some dimensions, e.g., OneIG-EN Diversity 0.177 and GenEval Position 0.72 versus other AR models.

Experimental setup

Datasets: DataComp-1B ImageNet-1K LAION Seedream-generated images Z-Image-Turbo-generated images

Benchmarks: ImageNet 256x256 GenEval DPG-Bench OneIG-EN OneIG-ZH TIIF Bench testmini

Metrics: FID IS Precision Recall PSNR SSIM GenEval overall DPG-Bench overall OneIG overall TIIF overall

Baselines: MAR SphereAR xAR LlamaGen RandAR RAR VAR PAR NAR NextStep-1 GLM-Image Janus-Pro

Builds on: LFQ (lookup-free quantization) MAR Rectified Flow MAGVIT-v2 RandAR Qwen3

Closest papers by meaning

• Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesis (CVPR 2025 · similarity 0.89)
• LANTERN: Accelerating Visual Autoregressive Models with Relaxed Speculative Decoding (ICLR 2025 · similarity 0.89)
• Parallelized Autoregressive Visual Generation (CVPR 2025 · similarity 0.89)
• Distilled Decoding 1: One-step Sampling of Image Auto-regressive Models with Flow Matching (ICLR 2025 · similarity 0.89)
• Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction (NeurIPS 2024 · similarity 0.88)
• Hierarchical Masked Autoregressive Models with Low-Resolution Token Pivots (ICML 2025 · similarity 0.88)