Research map / Probabilistic and scientific modeling / Molecules and proteins

Position: Compositional Generative Modeling: A Single Model is Not All You Need

ICML 2024 · Probabilistic and scientific modeling › Molecules and proteins · approach: Sparse concepts and hypothesis testing

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Position paper arguing that large generative systems should be built by composing smaller generative models, illustrated with toy experiments, planning, constraint satisfaction, and unsupervised component discovery, plus sampling methods for composed energy-based and diffusion models.

Problem

Large monolithic generative models are data-hungry, costly to train and adapt, and generalize poorly to unseen combinations of variables and tasks; the paper asks whether composing smaller generative models can address this.

Approach

Construct complex generative systems compositionally by multiplying or mixing separately learned factors (energy-based or diffusion-based), enabling data-efficient learning, generalization to unseen combinations, zero-shot task construction by composition, and discovery of components from data.

Key results

• On MATH, composing 5 GPT-3.5 instances reportedly reaches 58.0 ± 2.8% accuracy, above GPT-4 at 55.0 ± 2.9% (cited from prior work).
• On Maze2D, a factorized trajectory model more accurately simulates dynamics from very few agent episodes and trains in fewer iterations than a monolithic model (qualitative figure-based comparison).
• Toy 2D experiments show that composing two factor models is more data-efficient than one monolithic model trained on the product or mixture with equal total data.

Measured effects reported in the paper

Surprising findings

• Composing five GPT-3.5 instances reportedly outperforms GPT-4 on MATH, suggesting composition can substitute for scale.

Limitations

• Current compositional structure is prespecified, limiting generalization to new tasks.
• Discovering components assumes data factorizes into independent components; spurious correlations can break this.
• Evidence is largely illustrative and drawn from prior work rather than systematic new experiments.

Experimental setup

Datasets: Maze2D MATH ImageNet CLEVR-style cubes and spheres scenes

Benchmarks: Maze2D MATH

Metrics: accuracy trajectory accuracy vs. number of episodes training iterations

Baselines: monolithic single-model training GPT-4 GPT-3.5 DALL-E 3 GPT-4V

Builds on: energy-based models diffusion models probabilistic graphical models compositional visual generation (Du et al.) Diffuser

Closest papers by meaning

• Compositional Generalization via Forced Rendering of Disentangled Latents (ICML 2025 · similarity 0.88)
• Does Data Scaling Lead to Visual Compositional Generalization? (ICML 2025 · similarity 0.87)
• Scaling can lead to compositional generalization (NeurIPS 2025 · similarity 0.87)
• When does compositional structure yield compositional generalization? A kernel theory. (ICLR 2025 · similarity 0.87)
• Limits of Transformer Language Models on Learning to Compose Algorithms (NeurIPS 2024 · similarity 0.86)
• In-Context Compositional Learning vis Sparse Coding Transformer (NeurIPS 2025 · similarity 0.86)