Research map / Image generation / Autoregressive and latent generation
FlexTok: Resampling Images into 1D Token Sequences of Flexible Length
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces FlexTok, a 1D image tokenizer that resamples 2D images into ordered, variable-length (1 to 256) discrete token sequences using nested dropout, causal register masks, and a rectified flow decoder with REPA, and shows AR generation on ImageNet and text-to-image with coarse-to-fine token use.
Problem
Fixed-length 2D grid and 1D tokenizers encode every image into the same number of tokens regardless of content complexity, so conditional AR generators must always produce the full sequence even for simple conditions.
Approach
Resample image VAE latents into 1D register tokens with causal attention among registers and nested dropout so tokens are ordered coarse-to-fine, and decode any prefix with a rectified flow decoder conditioned on the quantized tokens plus REPA alignment to DINOv2-L features.
Key results
- FlexTok d18-d28 with 1.33B AR model achieves gFID 1.86 on ImageNet-1k class-conditional generation at up to 256 tokens, versus TiTok-S 1.97 at 128 tokens.
- FlexTok reconstructs plausible images with a single token; rFID is reported across token counts from 1 to 256 on ImageNet validation.
- Linear probing on FlexTok register tokens reaches 64.6% top-1 ImageNet accuracy with performance increasing monotonically with more tokens.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the 2D grid with 1D register tokenization (ablation) | FlexTok d12-d12 with a rectified flow decoder · ImageNet-1k validation | rFID: 32.85 -> 23.28 | Helped |
| Replace MSE decoder loss with rectified flow (ablation) | 2D grid tokenizer · ImageNet-1k validation | gFID: 35.93 -> 23.29 | Helped |
| Replace MSE decoder loss with rectified flow (ablation) | 2D grid tokenizer · ImageNet-1k validation | MAE: 0.059 -> 0.083 | Hurt |
| Add REPA loss (ablation) | FlexTok d12-d12 with 1D register tokens and rectified flow · ImageNet-1k validation | gFID: 22.16 -> 7.40 | Helped |
| Use logit-normal rather than mode(0.25) noise (ablation) | FlexTok d12-d12 with 1D register tokens and rectified flow · ImageNet-1k validation | rFID: 23.28 -> 19.03 | Helped |
| Add a causal attention mask to register tokens (ablation) | FlexTok d12-d12 with REPA · ImageNet-1k validation, 256 tokens | gFID: 7.40 -> 5.04 | Helped |
| Add uniform nested dropout (ablation) | FlexTok d12-d12 with REPA and causal register mask · ImageNet-1k validation, 256 tokens | rFID: 3.28 -> 4.01 | Hurt |
| Use Pow2 rather than uniform nested dropout (ablation) | FlexTok d12-d12 with REPA and causal register mask · ImageNet-1k validation | rFID: 19.97 -> 10.39 (4 tokens); 4.01 -> 7.22 (256 tokens) | Mixed |
| Use Unifpow2 rather than uniform nested dropout (ablation) | FlexTok d12-d12 with REPA and causal register mask · ImageNet-1k validation | rFID: 19.97 -> 30.17 (4 tokens); 4.01 -> 3.52 (256 tokens) | Mixed |
| Increase VAE latent channels from 4 to 16 (ablation) | SDXL VAE architecture · COCO-30k validation | rFID: 1.53 -> 0.35 | Helped |
| Generate more tokens | FlexTok class-conditional autoregressive generation · ImageNet-1k | DINOv2-L classification accuracy | Helped |
| Generate more tokens | FlexTok text-conditional autoregressive generation · COCO 30k validation | CLIPScore | Helped |
Surprising findings
- Generating as few as 8 to 16 tokens already satisfies ImageNet class conditioning, while text-conditioned generation benefits from the full 256 tokens.
- REPA improves perceptual reconstruction and generation metrics (rFID 23.28 to 5.98) but does not speed up convergence of the reconstruction loss itself.
Limitations
- Text-dense or structured content such as text in images requires more tokens and objectives that prioritize semantics.
- Decoder adaLN parameters make up half of decoder parameters, adding parameter cost without much FLOP cost.
- At long sequences (>128 tokens) AR model size strongly matters, so the flow decoder cannot compensate for a weak AR model.
Experimental setup
Datasets: ImageNet-1k DFN-2B COCO 30k
Benchmarks: ImageNet-1k COCO 30k
Metrics: rFID gFID MAE DreamSim CLIPScore DINOv2-L top-1 accuracy linear probing accuracy
Baselines: TiTok-L TiTok-B TiTok-S LlamaGen Open MAGVIT-v2 MaskGIT VQ-GAN Taming VQ-GAN ElasticTok ALIT 2D grid tokenizer with flow decoder
Builds on: TiTok LlamaGen REPA nested dropout SDXL VAE
Closest papers by meaning
- Adaptive Length Image Tokenization via Recurrent Allocation (ICLR 2025 · similarity 0.90)
- An Image is Worth 32 Tokens for Reconstruction and Generation (NeurIPS 2024 · similarity 0.90)
- Highly Compressed Tokenizer Can Generate Without Training (ICML 2025 · similarity 0.89)
- ImageFolder: Autoregressive Image Generation with Folded Tokens (ICLR 2025 · similarity 0.89)
- When Worse is Better: Navigating the Compression Generation Trade-off In Visual Tokenization (NeurIPS 2025 · similarity 0.89)
- Language-Guided Image Tokenization for Generation (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.