Research map / Image generation / Autoregressive and latent generation
Factorized Diffusion Autoencoder for Unsupervised Disentangled Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FDAE, a content-mask factorization network jointly trained with a diffusion-based conditional generator for unsupervised disentangled representation learning, using content decorrelation and mask entropy losses.
Problem
Unsupervised disentangled representation learning typically relies on independence or informativeness assumptions and ignores visual interpretability of factors, so learned factors often fail to align with human-defined concepts.
Approach
Factorize an image's latent code into N groups of "content + mask" pairs whose combined content masks condition a diffusion autoencoder decoder; decorrelate content codes in latent space and masks in spatial space with dedicated losses.
Key results
- On Shapes3d FDAE reaches DCI 0.917 vs 0.884 for the best prior method (VCT), and on MPI3D-real-complex DCI 0.644 vs baseline DiffAE 0.506.
- On Market-1501 attribute prediction FDAE averages 71.6% accuracy vs 66.2% for DiffAE and 65.7% for VCT among unsupervised methods.
- Ablation on MPI3D shows CMFNet raises DCI from 0.506 (DiffAE baseline) to 0.631, and the full model with both losses reaches 0.644.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add content-mask factorization to the encoder-generator baseline (ablation) | FDAE · MPI3D-real-complex | DCI: 0.506 -> 0.631 | Helped |
| Add content-mask factorization to the encoder-generator baseline (ablation) | FDAE · MPI3D-real-complex | MIG: 0.059 -> 0.152 | Helped |
| Add content decorrelation loss (ablation) | FDAE with CMFNet · MPI3D-real-complex | MIG: 0.152 -> 0.161 | Hurt |
| Add content decorrelation loss (ablation) | FDAE with CMFNet · MPI3D-real-complex | DCI: 0.631 -> 0.642 | Helped |
| Add mask entropy loss (ablation) | FDAE with CMFNet · MPI3D-real-complex | DCI: 0.631 -> 0.637 | Helped |
| Add mask entropy loss (ablation) | FDAE with CMFNet · MPI3D-real-complex | MIG: 0.152 -> 0.180 | Helped |
| Use both content decorrelation and mask entropy losses (ablation) | FDAE with CMFNet · MPI3D-real-complex | DCI: 0.631 -> 0.644 | Helped |
| Use both content decorrelation and mask entropy losses (ablation) | FDAE with CMFNet · MPI3D-real-complex | MIG: 0.152 -> 0.197 | Helped |
| Set concept number N to 2 rather than 1 (ablation) | FDAE · MPI3D-real-complex | DCI: 0.506 -> 0.437 | Hurt |
| Set concept number N to 2 rather than 1 (ablation) | FDAE · MPI3D-real-complex | MIG: 0.059 -> 0.221 | Helped |
| Increase concept number from N=6 to N=10 (ablation) | FDAE · MPI3D-real-complex | DCI: 0.644 -> 0.669 | Helped |
| Increase concept number from N=6 to N=10 (ablation) | FDAE · MPI3D-real-complex | MIG: 0.197 -> 0.074 | Hurt |
Surprising findings
- MIG on Shapes3d and Cars3d is lower than VAE/GAN methods because vector-wise representations are penalized by the scalar-wise MIG metric.
Limitations
- Some learned concepts are not meaningful (e.g., concepts 5 and 6 on MPI3D), and MIG is weaker than scalar-wise methods on Shapes3d and Cars3d.
- Requires selecting the number of concepts N via a self-MIG sweep.
Experimental setup
Datasets: Shapes3d MPI3D-real-complex Cars3d Market-1501 FFHQ
Benchmarks: Shapes3d MPI3D Cars3d Market-1501
Metrics: DCI disentanglement FactorVAE score MIG attribute prediction accuracy
Baselines: FactorVAE beta-TCVAE DisCo-VAE DAVA InfoGAN-CR LatentDiscovery ClosedForm GANSpace DeepSpectral DisCo-GAN VCT DiffAE
Builds on: DiffAE EDM DCI COMET beta-VAE
Closest papers by meaning
- Diffusion Model with Cross Attention as an Inductive Bias for Disentanglement (NeurIPS 2024 · similarity 0.89)
- Disentangled Sparse Representations for Concept-Separated Diffusion Unlearning (NeurIPS 2026 · similarity 0.89)
- Graph-based Unsupervised Disentangled Representation Learning via Multimodal Large Language Models (NeurIPS 2024 · similarity 0.88)
- Disentanglement Beyond Static vs. Dynamic: A Benchmark and Evaluation Framework for Multi-Factor Sequential Representations (NeurIPS 2025 · similarity 0.88)
- Disentangled Representation Learning via Modular Compositional Bias (NeurIPS 2025 · similarity 0.88)
- Disentangled Representation Learning via Flow Matching (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.