Research map / Probabilistic and scientific modeling / Molecules and proteins

Nonparametric Identification of Latent Concepts

ICML 2025 · Probabilistic and scientific modeling › Molecules and proteins · approach: Probabilistic embeddings and relational inference

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proves nonparametric identifiability of latent concepts from multiple classes of observations by comparing classes (pairwise/local and global structural-diversity conditions), without assuming concept types, functional forms, or parametric generators.

Problem

Concept learning from observations lacks general theoretical identifiability guarantees; prior work requires parametric assumptions (linearity, additivity, sparsity, object-centric restrictions). The paper asks which concepts can be recovered reliably in a general nonparametric setting.

Approach

Uses class-conditional comparison as the source of identifiability: pairwise comparisons disentangle each class's unique concepts via the Jacobian of the generator w.r.t. class variables, a structural diversity assumption extends this to element-wise identifiability of all class-dependent concepts, and two-class distributional variability identifies class-independent concepts block-wise.

Key results

• On synthetic data, the theory-conforming models achieve higher MCC than the Base model with lower variance across 10 random trials for both class-dependent and all-concept settings.
• On Fashion-MNIST, AnimalFace, Flower102 and FFHQ, identified concepts are visually interpretable and align with class semantics (e.g., sleeve length, heel height, ursid/monochrome for panda).
• Pairwise comparison requires only two classes and class-independent concepts are identified block-wise under two-class distributional variability.

Measured effects reported in the paper

Surprising findings

• Two domains or classes suffice for block-wise identifiability of class-independent concepts, fewer than the n_A+1 or 2n_A+1 required by prior proof techniques.

Limitations

• Global identifiability requires the structural diversity assumption, which fails when classes share all concepts (e.g., dog breeds with only shared traits); partial identifiability is then only local.
• Experiments are small-scale image settings and the estimator relies on regularized maximum likelihood with l1 penalties, with no large-scale or foundation-model validation.
• Identifiability guarantees are asymptotic and rely on sufficient diversity and smooth, positive densities and diffeomorphic mixing.

Experimental setup

Datasets: synthetic data Fashion-MNIST EMNIST AnimalFace Flower102 FFHQ

Benchmarks: Fashion-MNIST EMNIST AnimalFace Flower102 FFHQ

Metrics: Mean Correlation Coefficient (MCC)

Baselines: Base model without structural conditions

Builds on: nonlinear ICA identifiability sparsity-based identifiability of latent variable models object-centric learning identifiability causal representation learning

Closest papers by meaning

• Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Space (NeurIPS 2024 · similarity 0.87)
• Concept Modulation Models: A Unified Framework for Identifiability and Extrapolation (NeurIPS 2026 · similarity 0.87)
• Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometry (NeurIPS 2025 · similarity 0.87)
• From Causal to Concept-Based Representation Learning (NeurIPS 2024 · similarity 0.87)
• Learning Discrete Concepts in Latent Hierarchical Models (NeurIPS 2024 · similarity 0.87)
• Interpretable but Fragile? Robustness of Concept Bottlenecks under Geometric-Semantic Perturbations (NeurIPS 2026 · similarity 0.86)