Research map / Label-efficient and robust learning / Recognition and long-tail
Towards Compositionality in Concept Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Compositional Concept Extraction (CCE), an unsupervised method that finds concept subspaces per attribute that are mutually orthogonal while allowing within-attribute correlation, yielding concept representations that compose additively.
Problem
Unsupervised concept extraction from foundation model embeddings yields concepts that are discriminative but do not compose additively (e.g., white + small does not give small white birds), limiting uses like explanation, editing, and concept-based classification.
Approach
Identify two properties of compositional concepts (concepts from different attributes are roughly orthogonal; concepts within an attribute may be correlated) and extract concepts attribute-by-attribute by learning a subspace per attribute via silhouette-based optimization, then removing that subspace by orthogonal projection before finding the next attribute.
Key results
- On controlled settings CCE achieves the lowest compositionality score on CUB-sub (0.459) and Truth-sub (3.689), beating all unsupervised baselines and matching ground truth.
- CCE's average cosine similarity to ground-truth concepts is 0.992 on CLEVR, 0.770 on CUB-sub, and 0.804 on Truth-sub, above all baselines.
- In the full setting CCE eventually outperforms unsupervised baselines in downstream classification as the number of concepts grows, and beats no-concept embeddings by a large margin on CUB and HAM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CCE instead of ACE | concept extraction with CLIP · CLEVR | compositionality score: 3.474 -> 3.163 | Helped |
| Use CCE instead of PCA | concept extraction with CLIP · CUB-sub | compositionality score: 0.472 -> 0.459 | Helped |
| Use CCE instead of DictLearn | concept extraction with Llama-2 13B Chat · Truth-sub | compositionality score: 3.708 -> 3.689 | Helped |
| Use CCE instead of DictLearn | concept extraction with CLIP · CLEVR | cosine similarity to ground truth: 0.745 -> 0.992 | Helped |
| Use CCE instead of ACE | concept extraction with CLIP · CUB-sub | cosine similarity to ground truth: 0.719 -> 0.770 | Helped |
| Use CCE instead of DictLearn | concept extraction with Llama-2 13B Chat · Truth-sub | cosine similarity to ground truth: 0.686 -> 0.804 | Helped |
Surprising findings
- NMF reaches ROC-AUC up to 0.907 on CLEVR yet its extracted concepts are not compositional, showing discriminability does not imply composability.
Limitations
- Assumes a flat concept structure and does not handle non-compositional or hierarchical concepts (e.g., hot dog).
- Discovered concept vectors have no names; naming requires manual inspection of top-scoring samples.
- Theoretical guarantees rely on idealized assumptions such as spherical Gaussian representations in high dimension.
Experimental setup
Datasets: CLEVR CUB HAM10000 Truth News
Benchmarks: CLEVR CUB HAM10000 Truth News
Metrics: MAP of compositional prediction compositionality score cosine similarity to ground-truth concepts ROC-AUC downstream classification accuracy
Baselines: PCA NMF ACE (KMeans) Dictionary Learning Concept Transformer Random
Builds on: spherical k-means silhouette score concept bottleneck models Concept Transformer
Closest papers by meaning
- ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Models (AAAI 2024 · similarity 0.86)
- ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficulty (NeurIPS 2024 · similarity 0.86)
- Does Data Scaling Lead to Visual Compositional Generalization? (ICML 2025 · similarity 0.85)
- Not Only Text: Exploring Compositionality of Visual Representations in Vision-Language Models (CVPR 2025 · similarity 0.84)
- Restyling Unsupervised Concept Based Interpretable Networks with Generative Models (ICLR 2025 · similarity 0.84)
- ICE: Intrinsic Concept Extraction from a Single Image via Diffusion Models (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.