Research map / Probabilistic and scientific modeling / Causal discovery
Learning Discrete Concepts in Latent Hierarchical Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Formalizes concept learning as identifying a discrete latent hierarchical causal model from continuous high-dimensional observations, gives identifiability conditions, and interprets latent diffusion models as hierarchical concept learners with synthetic and diffusion experiments.
Problem
Under what conditions can discrete latent concepts and their hierarchical causal structure be identified from unsupervised high-dimensional continuous observations such as images?
Approach
Model concepts as discrete latent causal variables in a hierarchical DAG with continuous observed leaves; identify discrete components via manifold disjointness (invertible generating function, connected continuous support) and then recover the hierarchy using non-negative rank / t-separation rank tests, extending graph search beyond trees and multi-level DAGs.
Key results
- On 9 synthetic hierarchical graphs, the proposed method achieves F1 of about 0.93-0.98 versus 0.63-0.69 for the baseline.
- On 7 purely discrete graphs, the method attains F1 of 0.73-1.0 while the baseline ranges 0.0-0.63.
- Diffusion experiments show concept-level dependence via prompt interventions and that early diffusion steps control global attributes (breeds) while late steps control details (collar, shirt patterns); these are qualitative/supporting, not quantitative benchmarks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Ours instead of Baseline | Synthetic discrete hierarchical model identification · Graphs 1–9 (synthetic hierarchical models) | F1: Graph 1: 0.67 $\pm$ 0.0 -> 0.94 $\pm$ 0.1; Graph 2: 0.69 $\pm$ 0.1 -> 0.98 $\pm$ 0.1; Graph 3: 0.67 $\pm$ 0.0 -> 0.94 $\ | Helped |
| Use Ours instead of Baseline | Synthetic discrete hierarchical model identification · Graphs 1–7 (synthetic purely discrete models) | F1: Graph 1: 0.24 $\pm$ 0.3 -> 1.0 $\pm$ 0.0; Graph 2: 0.48 $\pm$ 0.0 -> 1.0 $\pm$ 0.0; Graph 3: 0.33 $\pm$ 0.2 -> 0.73 $\pm | Helped |
Surprising findings
- Baseline methods fail completely (F1 0.0) on some graphs, showing that handling higher-level latent variables is essential.
Limitations
- Theory does not directly guarantee diffusion model success; interpretation of latent diffusion is intuitive.
- Identification algorithm can be expensive for large graphs due to dependence on the number of discrete states.
- Continuous attributes are left unidentified and identification of the continuous part is future work.
Experimental setup
Datasets: synthetic Gaussian mixture data
Benchmarks: synthetic hierarchical graphs (9 graphs, F1) synthetic purely discrete graphs (7 graphs, F1)
Metrics: F1 score
Baselines: discrete latent identification baseline (mixture-oracle method)
Builds on: linear latent hierarchical model identification mixture oracle discrete identification (tensor decomposition) latent tree / multi-level DAG identification t-separation
Closest papers by meaning
- From Causal to Concept-Based Representation Learning (NeurIPS 2024 · similarity 0.91)
- Differentiable Causal Discovery for Latent Hierarchical Causal Models (ICLR 2025 · similarity 0.88)
- CASL: Concept-Aligned Sparse Latents for Interpreting Diffusion Models (NeurIPS 2026 · similarity 0.88)
- Probing the Latent Hierarchical Structure of Data via Diffusion Models (ICLR 2025 · similarity 0.88)
- Towards Identifiability of Hierarchical Temporal Causal Representation Learning (NeurIPS 2025 · similarity 0.88)
- Neural Causal Abstractions (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.