Research map / Image generation / Autoregressive and latent generation

DC-SAE: Deep Compression Semantic Autoencoder for Faster Diffusion Convergence

NeurIPS 2026 · Image generation › Autoregressive and latent generation · approach: Latent diffusion with learned tokenizers · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DC-SAE, a dual-stream autoencoder that pairs a frozen DINOv2 semantic branch (pre-merge resized to 32x) with a trainable pixel branch and a ViT decoder with Spatial DeMerger, to get high compression with fast diffusion convergence.

Problem

High-compression (32x) tokenizers speed up generation but slow diffusion convergence and reduce reconstruction fidelity; semantic autoencoders (RAE-style) lose reconstruction quality when pushed to 32x compression.

Approach

Adapt a frozen semantic encoder to 32x by resizing the input before patchifying (pre-merge), and jointly train an unconstrained pixel encoder whose latent is concatenated with the semantic latent and decoded by a ViT decoder with a Spatial DeMerger that expands tokens only on the decoder side.

Key results

• At ImageNet 512x512 with 32x compression, DC-SAE reaches 29.79 PSNR and 3.37 gFID without CFG, versus 26.25 PSNR and 7.47 gFID for DC-AE (SiT-XL).
• At ImageNet 256x256, DC-SAE achieves 0.45 rFID and 3.31 gFID (DiT-XL with DDT wide head), with IS 180.71.
• Text-to-image 1.6B DiT with DC-SAE at 1024x1024 with CFG reaches 0.84 GenEval and 86.007 DPG-Bench; DC-SAE throughput is 4.41x-4.75x that of DC-AE across resolutions.

Surprising findings

• Masking the semantic branch at decoding has little effect on reconstruction, while masking the pixel branch causes severe degradation.
• Late concatenation of a separately trained pixel autoencoder gives better reconstruction (PSNR 31.91) but worse generation (80-epoch FID 5.64) than jointly trained DC-SAE.

Limitations

• Text-to-image evaluated only with one 1.6B DiT configuration and partly internal data; broader validation not done.
• Lacks a full understanding of which semantic encoder properties matter for generation.
• Pre-merge vs post-merge preference is empirical without principled explanation.

Experimental setup

Datasets: ImageNet-1K LAION-COCO internal text-image data

Benchmarks: ImageNet 256x256 ImageNet 512x512 GenEval DPG-Bench

Metrics: PSNR rFID gFID Inception Score GenEval DPG-Bench throughput

Baselines: RAE SD-VAE VA-VAE SVG REPA DC-AE DC-AE-1.5 TC-AE USiT SiT-XL DC-Gen-FLUX.1-Krea-12B

Builds on: RAE DC-AE DDT REPA

Closest papers by meaning

• Deep Compression Autoencoder for Efficient High-Resolution Diffusion Models (ICLR 2025 · similarity 0.92)
• Masked Autoencoders Are Effective Tokenizers for Diffusion Models (ICML 2025 · similarity 0.90)
• Boosting Generative Image Modeling via Joint Image-Feature Synthesis (NeurIPS 2025 · similarity 0.90)
• Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models (CVPR 2025 · similarity 0.90)
• One-Step Diffusion-Based Image Compression with Semantic Distillation (NeurIPS 2025 · similarity 0.89)
• Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Think (NeurIPS 2025 · similarity 0.89)