Research map / Label-efficient and robust learning / Recognition and long-tail
Compositional Generalization via Forced Rendering of Disentangled Latents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that disentangled latents fail to yield compositional generalization in a 2D Gaussian bump decoder task due to manifold warping and memorization, and that forcing latents to render into pixel space (low-rank embedding regularization or stripe-based data curriculum) enables OOD composition.
Problem
Disentangled or factorized representations are assumed to support compositional generalization, but empirical results are mixed; generative decoders often fail to compose factors for out-of-distribution (OOD) samples.
Approach
Failure arises because deeper layers re-entangle factorized inputs and memorize training data; compositional generalization is recovered when disentangled factors are rendered directly in the output (pixel) space, either via low-rank/entropy/variance-regularized 2D embedding filters or via a curriculum of 1D stripe images.
Key results
- Networks with disentangled bump or ramp inputs fail in the OOD region, generating multiple bumps, with a superposition-of-memorized-data mechanism.
- Entropy+variance regularization on low-rank 2D embedding filters gives the best OOD MSE while keeping low ID error in the ablation study.
- Stripes-only training generalizes zero-shot to additive stripe conjunctions, and stripes+bumps data needed to reach 90% accuracy scales linearly in N versus about N^3 for bumps-only.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use disentangled bump or ramp input encodings with a standard CNN decoder | CNN decoder for 2D Gaussian bump generation · Central OOD region | OOD generation error | Hurt |
| Use disentangled 1-hot or positional input encodings with a standard decoder | Decoder-only architecture for 2D Gaussian bump generation · OOD region | OOD compositional generalization | Hurt |
| Add low-rank embedding regularization using entropy and variance penalties | Factorized embedding architecture · OOD region, including corner OOD | MSE | Helped |
| Add entropy regularization only (ablation) | Factorized embedding architecture · OOD region | MSE | Helped |
| Add variance regularization only (ablation) | Factorized embedding architecture · OOD region | MSE | Helped |
| Use entropy and variance regularization together instead of no regularization (ablation) | Factorized embedding architecture · ID region | MSE | Mixed |
| Train on independent horizontal and vertical stripes, with no 2D bump examples | CNN for 2D Gaussian bump generation · OOD region | Compositional generation | Helped |
| Add stripes to bump training data | CNN for 2D Gaussian bump generation · OOD region | Data required to reach 90% accuracy: ∼ N^3 -> linear with N | Helped |
| Increase the number of 2D bump examples in the stripes training set (ablation) | CNN for 2D Gaussian bump generation · OOD region | Bump generation | Helped |
Surprising findings
- Disentangled input encodings (bump, ramp, 1-hot, positional) fail to compositionally generalize, showing disentangled inputs alone are insufficient.
- Models trained only on stripes generate additive stripe conjunctions in the OOD region without seeing any compositional examples.
Limitations
- Results are on a toy synthetic 2D Gaussian bump task; no evaluation on large-scale models or real datasets.
- Proposed approaches (low-rank regularization, stripe curriculum) are not validated beyond the toy setting, and the authors state they do not claim universal applicability.
Experimental setup
Datasets: synthetic 2D Gaussian bump images 1D Gaussian stripe images MNIST (rotation follow-up experiments)
Benchmarks: synthetic 2D Gaussian bump OOD generation
Metrics: MSE in-distribution vs OOD linear probe R^2 factorization score from metric tensor volume metric log10(dv) data scaling to 90% location accuracy
Baselines: No Reg Entropy Only Variance Only bumps-only training
Builds on: beta-VAE FactorVAE
Closest papers by meaning
- When does compositional structure yield compositional generalization? A kernel theory. (ICLR 2025 · similarity 0.90)
- Does Data Scaling Lead to Visual Compositional Generalization? (ICML 2025 · similarity 0.88)
- Position: Compositional Generative Modeling: A Single Model is Not All You Need (ICML 2024 · similarity 0.88)
- Disentangled Representation Learning via Modular Compositional Bias (NeurIPS 2025 · similarity 0.87)
- Fully Distributed, Flexible Compositional Visual Representations via Soft Tensor Products (NeurIPS 2024 · similarity 0.87)
- How Diffusion Models Learn to Factorize and Compose (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.