Research map / Image generation / Autoregressive and latent generation

Efficient Image Synthesis with Sphere Latent Encoder

NeurIPS 2026 · Image generation › Autoregressive and latent generation · approach: Generative latent space tooling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Decouples Sphere Encoder into a frozen pretrained RAE (DINOv2-B encoder, ViT-XL decoder) plus a SiT denoiser trained only in spherical latent space, enabling few-step generation without repeated pixel-latent transitions.

Problem

Sphere Encoder produces few-step images but alternates between pixel and latent space at every step and jointly optimizes reconstruction and generation in one architecture, causing high computational overhead and objective conflict.

Approach

Keep a fixed pretrained representation autoencoder and train a separate SiT denoiser on hyperspherically projected latents with reconstruction and consistency losses in latent space only; decode to pixels once at the end, with logit-normal noise sampling and no latent consistency loss.

Key results

• On ImageNet-1K at 4x2 NFE, the method reaches FID 2.25 versus 4.02 for Sphere Encoder (675M vs 1.3B params).
• On Animal-Faces at 4 steps, FID 6.89 vs 18.23 for Sphere Encoder, with about 85% lower inference FLOPs (390 vs 4554 GFLOPs).
• At 6x2 NFE on ImageNet-1K the method reaches FID 2.11, competitive with multi-step diffusion models requiring 250x2 NFE.

Surprising findings

• Adding the latent consistency loss from Sphere Encoder degraded performance in the decoupled setting (FID 4.82 vs 4.68 without it on ImageNet-100).

Limitations

• Evaluated only on class-conditional generation; text-to-image not tested.
• Relies on a strong pretrained representation autoencoder (DINOv2-based), which may limit generality.
• Fixed encoder makes high-quality one-step sampling challenging.

Experimental setup

Datasets: Animal-Faces Oxford-Flowers ImageNet-1K

Benchmarks: Animal-Faces Oxford-Flowers ImageNet-1K

Metrics: gFID CMMD IS Precision Recall GFLOPs

Baselines: Sphere Encoder MeanFlow alpha-Flow iMF DiT-XL/2 SiT-XL/2 REPA REG LightningDiT REPA-E GAE RAE + DiT-XL

Builds on: Sphere Encoder Representation Autoencoder (RAE) SiT RMSNorm

Closest papers by meaning

• SphereDiffusion: Spherical Geometry-Aware Distortion Resilient Diffusion Model (AAAI 2024 · similarity 0.86)
• Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Think (NeurIPS 2025 · similarity 0.86)
• SP$^3$: Spherical Priors for Plug-and-Play Restoration (NeurIPS 2026 · similarity 0.85)
• Kepler codebook (ICML 2024 · similarity 0.84)
• LMD: Faster Image Reconstruction with Latent Masking Diffusion (AAAI 2024 · similarity 0.84)
• Epsilon-VAE: Denoising as Visual Decoding (ICML 2025 · similarity 0.84)