Research map / Probabilistic and scientific modeling / Molecules and proteins
Enriching Disentanglement: From Logical Definitions to Quantitative Metrics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Converts logical definitions of disentanglement properties (modularity, informativeness) into quantitative metrics by replacing equality with strict premetrics, connectives with quantitative operations, and quantifiers with aggregators, then evaluates the metrics on controlled hand-defined functions.
Problem
Existing disentanglement metrics lack clear theoretical grounding about what properties they quantify, and it is unclear how to optimize them directly; metrics often lack proof that a learning method optimizes them.
Approach
A compositional conversion from higher-order equational predicates to real-valued quantities: equality becomes a strict premetric, the Heyting algebra of truth values becomes a quantale of extended non-negative reals (conjunction to addition, disjunction to min, implication to truncated subtraction), and quantifiers become aggregators (sup, mean, mean square, etc.). Derived metrics have guarantees
Key results
- Metrics derived from logical definitions give a perfect score (1.00) on the functions that truly satisfy the target property, e.g., redundancy, contraction, and constant functions for modularity and informativeness metrics.
- Variance, diameter, MPD, and max/mean contraction are differentiable and can serve as learning objectives, whereas radius, MAD, ME, MAE, and MSE require inner optimization and are not differentiable.
- Existing metrics such as MIG and DCI variants often entangle modularity and informativeness, e.g., MIG scores 0.00 on complement and misalignment.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Measure modularity with a proposed metric derived from the product-function definition | disentangled representations · controlled synthetic representations | modularity score | Helped |
| Measure informativeness with a proposed metric derived from retraction or contraction | disentangled representations · controlled synthetic representations | informativeness score | Helped |
| Use variance, diameter, MPD, or max/mean contraction instead of metrics requiring an inner optimization problem | proposed disentanglement metrics · controlled synthetic representations | differentiability | Helped |
| Use retraction approximation with linear models | proposed informativeness metrics · entanglement; nonlinear | informativeness score: 0.99; 0.99 | Hurt |
| Use a single existing metric score instead of evaluating modularity and informativeness separately | disentangled representations · controlled synthetic representations | property isolation | Hurt |
Surprising findings
- Linear-model-based retraction approximation falls short of perfect scores on some functions that should be perfect, due to limited expressiveness of the approximators.
- Injectivity-based contraction metric is zero only for non-contractive maps, a strictly stronger condition than injectivity (e.g., y -> 0.01y is injective but gets nonzero score).
Limitations
- Implication is only subhomomorphic, so the truncated subtraction sacrifices logical equivalence for continuity.
- Variance-based metrics are more susceptible to outliers and noise than robust alternatives.
- Experiments use directly defined functions rather than learned representations on datasets.
Experimental setup
Benchmarks: controlled synthetic function set (entanglement, rotation, duplicate, complement, misalignment, redundancy, contraction,
Metrics: radius MAD variance diameter MPD ME MAE MSE contraction Beta
Baselines: Beta-VAE metric Factor VAE metric MIG DCI Pair metric
Builds on: DCI disentanglement metric Hausdorff distance Lawvere metric spaces elementary topos theory
Closest papers by meaning
- Theory and Evaluation Metrics for Learning Disentangled Representations (ICLR 2020 · similarity 0.91)
- Disentanglement Analysis with Partial Information Decomposition (ICLR 2022 · similarity 0.90)
- Disentanglement via Latent Quantization (NeurIPS 2023 · similarity 0.90)
- Benchmarks, Algorithms, and Metrics for Hierarchical Disentanglement (ICML 2021 · similarity 0.89)
- Graph-based Unsupervised Disentangled Representation Learning via Multimodal Large Language Models (NeurIPS 2024 · similarity 0.88)
- Transferring disentangled representations: bridging the gap between synthetic and real images (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.