Research map / Image generation / Autoregressive and latent generation

dRAE: Representation Autoencoder with Hyper-Spherical Codes

NeurIPS 2026 · Image generation › Autoregressive and latent generation · approach: Hierarchical VQ-VAE generation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Hyper-Spherical Quantization (HSQ), which routes codes by cosine similarity while keeping Euclidean commitment, to build a discrete Representation Autoencoder (dRAE) that scales to a 131K codebook without collapse and supports understanding and T2I generation.

Problem

Vector quantization of high-dimensional semantic encoder features (e.g., SigLIP2) suffers codebook collapse and plateaus beyond ~16K codes, blocking discrete visual tokenizers that serve both understanding and generation.

Approach

Decouple code assignment from magnitude by routing with cosine similarity and using a cosine codebook loss, while keeping an L2 commitment loss on unnormalized features so the decoder retains magnitude information; trained end-to-end with distillation from the frozen-teacher VFM in one stage.

Key results

• With a 131,072 codebook, dRAE reaches 0.42 rFID, 24.52 PSNR and 0.72 SSIM on ImageNet reconstruction, beating VQRAE (1.31 rFID at 16K codebook).
• dRAE achieves 0.63 GenEval overall and 80.58 DPG-Bench, the best among the listed discrete generators, with 12M training pairs.
• In the C2I ablation, HSQ at 65,536 codes reaches gFID 4.45 vs 5.51 for VQ at the same size.

Surprising findings

• Normalizing features to the unit sphere before the MLLM barely hurts understanding (~1% drop) but substantially degrades reconstruction (rFID 4.62 to 9.57), supporting magnitude as essential for reconstruction.
• Making both codebook and commitment losses cosine-based degrades reconstruction sharply (rFID 12.7 vs 3.02), indicating magnitude must be supervised.

Limitations

• Feature reconstruction (non-pixel proxy) remains much harder, with PSNR only 8.24 for HSQ.
• Results on understanding at 384 resolution are mixed versus VQRAE and Tar, with some benchmarks (e.g., MME-P) lower.
• SSIM numbers for baselines were re-computed by the authors due to an evaluation bug, making comparisons dependent on their correction.

Experimental setup

Datasets: BLIP3-o 36M images LLaVA-1.5 data LLaVA-OneVision Megalith-10M T2I-2M BLIP3-o-60K ImageNet-1K

Benchmarks: ImageNet-1K reconstruction GQA TextVQA MMBench-en MME-Perception SEED-Bench-Img POPE GenEval DPG-Bench ImageNet class-conditional generation

Metrics: rFID PSNR SSIM gFID Inception Score GenEval overall DPG-Bench overall accuracy on MLLM benchmarks

Baselines: VQRAE RAE SD-VAE VILA-U TokLIP Tar UniTok QLIP LlamaGen Janus Show-o Meissonic

Builds on: RAE VQRAE SimVQ VQ-VAE LLaVA-1.5

Closest papers by meaning

• IDEAL: In-DEpth ALignment Makes A Discrete Representation AutoEncoder (NeurIPS 2026 · similarity 0.87)
• Improved Baselines with Representation Autoencoders (NeurIPS 2026 · similarity 0.86)
• Finite Scalar Quantization: VQ-VAE Made Simple (ICLR 2024 · similarity 0.86)
• Vision as a Dialect: Unifying Visual Understanding and Generation via Text-Aligned Representations (NeurIPS 2025 · similarity 0.86)
• UniTok: a Unified Tokenizer for Visual Generation and Understanding (NeurIPS 2025 · similarity 0.85)
• Reading Images Like Texts: Sequential Image Understanding in Vision-Language Models (ICLR 2026 · similarity 0.85)