Research map / Image generation / Autoregressive and latent generation

Steering Optimisation Trajectories in Diffusion Representation Learning

NeurIPS 2026 · Image generation › Autoregressive and latent generation · approach: Latent diffusion with learned tokenizers · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Analyses optimisation trajectories of diffusion autoencoders for unsupervised disentanglement, identifies reconstruction vs disentanglement regimes, and proposes SteeringDRL (gated residual U-Net plus a log-SNR curriculum) that improves attribute and spatial (object-centric) disentanglement.

Problem

Diffusion autoencoders with identical objectives and architectures can reach similar reconstruction quality yet learn very different latent representations; the causes of this divergence during training are unclear.

Approach

Frames training as trajectories of reconstruction vs disentanglement that split into two regimes early on, attributes regime selection to U-Net skip shortcuts and early noise-level exposure, and steers training with gated residual skips and a widening log-SNR band curriculum.

Key results

• On Shapes3D with N=10 tokens, SteeringDRL reaches DCI 0.926 ± 0.043 and FactorVAE 0.991 ± 0.017 versus EncDiff's 0.828 ± 0.099 and 0.921 ± 0.093.
• On ClevrTex, SteeringDRL (50 steps DDIM) reaches mBO 69.69 and mIoU 68.31, above SlotDiffusion's 58.83 and 55.92 under the same conditions, with FID 26.67 vs 42.95.
• On PascalVOC with a ~138M-parameter U-Net and pretrained DINO ViT features, SteeringDRL improves over SlotDiffusion on all reported metrics (e.g., mBO_i 49.1 vs 47.2).

Surprising findings

• Zeroing skip connections improves disentanglement but sharply degrades reconstruction (LPIPS rises from 0.0025 to 0.157).
• Adding the log-SNR curriculum to the EncDiff baseline did not consistently improve performance; its benefit appeared mainly combined with the gated residual U-Net.

Limitations

• Regime separation is clearest only in low-capacity settings; larger models mask it.
• Zeroed-skip variants degrade reconstruction substantially, so the trade-off is not solved by removing skips.
• Hypotheses H1 and H2 are supported by ablations and analysis rather than a formal causal test.

Experimental setup

Datasets: Shapes3D Cars3D MPI3D-toy ClevrTex PascalVOC

Benchmarks: Shapes3D Cars3D MPI3D-toy ClevrTex PascalVOC

Metrics: DCI FactorVAE MIG LPIPS FG-ARI mIoU mBO FID

Baselines: EncDiff DisDiff FDAE DyGA Modular IB SlotDiffusion SLATE SLATE+ StableLSD MetaSlot DINOSAUR SA + DINO ViT

Builds on: EncDiff SlotDiffusion VDM++ EDM loss weighting SkipDropout

Closest papers by meaning

• Why DiT Models Underperform as Representation Learners without Long Skip Connections (NeurIPS 2026 · similarity 0.88)
• Learning Diffusion Models with Flexible Representation Guidance (NeurIPS 2025 · similarity 0.88)
• Tracing the Roots: Leveraging Temporal Dynamics in Diffusion Trajectories for Origin Attribution (NeurIPS 2025 · similarity 0.88)
• Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inference (NeurIPS 2024 · similarity 0.87)
• Diffusion Model with Cross Attention as an Inductive Bias for Disentanglement (NeurIPS 2024 · similarity 0.87)
• Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models (CVPR 2025 · similarity 0.87)