Research map / Label-efficient and robust learning / Recognition and long-tail
Disentangled Representation Learning with the Gromov-Monge Gap
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the Gromov-Monge gap (GMG), a debiased distortion regularizer based on quadratic optimal transport, and applies it to VAE decoders to improve unsupervised disentanglement on four image benchmarks.
Problem
Unsupervised disentangled representation learning via prior matching conflicts with geometry preservation: a map that exactly matches the prior while fully preserving distances or angles generally does not exist, so existing distortion penalties over-penalize and hurt disentanglement.
Approach
Regularize the VAE encoder or decoder with the Gromov-Monge gap, defined as the distortion of a map minus the minimal achievable Gromov-Wasserstein distortion between the reference distribution and its push-forward. This yields a debiased distortion that is zero when the map is a Gromov-Monge map and equals the distortion when distortion-free transport is possible. The gap is computed via entropic
Key results
- On Shapes3D, decoder GMG with Cos costs raises beta-TCVAE + HFS DCI-D from 90.2 (base) to 96.9, versus 91.9 for the distortion regularizer.
- On DSprites with Cos costs, decoder GMG lifts beta-VAE DCI-D from 26.2 to 39.5 and beta-TCVAE + HFS from 48.7 to 50.1 (distortion: 49.0).
- Gradients of the GMG show markedly higher alignment across batches than those of the distortion regularizer, indicating greater stability.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add decoder GMG with cosine-similarity costs | β-VAE and β-TCVAE-based models · Shapes3D | DCI-D: 76.8 ± 4.1 -> 82.1 ± 4.5; 81.3 ± 4.7 -> 83.7 ± 8.8; 87.5 ± 3.3 -> 95.7 ± 5.8; 91.9 ± 9.4 -> 96.9 ± 4.9 | Helped |
| Add decoder GMG with cosine-similarity costs | β-VAE and β-TCVAE-based models · DSprites | DCI-D: 28.6 ± 19.3 -> 39.5 ± 15.2; 32.4 ± 8.5 -> 42.2 ± 3.6; 39.3 ± 18.1 -> 46.7 ± 2.0; 49.0 ± 11.2 -> 50.1 ± 8.5 | Helped |
| Add decoder GMG with cosine-similarity costs | β-VAE and β-TCVAE-based models · SmallNORB | DCI-D: 28.2 ± 0.3 -> 28.3 ± 0.6; 29.9 ± 0.4 -> 29.9 ± 0.5 | No clear effect |
| Add decoder GMG with cosine-similarity costs | β-VAE and β-TCVAE-based models · Cars3D | DCI-D: 26.8 ± 3.6 -> 30.1 ± 5.6; 33.7 ± 4.2 -> 36.4 ± 5.7 | Helped |
| Use cosine-similarity costs instead of (scaled) squared-L2 costs (ablation) | decoder geometry regularization on Shapes3D · Shapes3D | DCI-D: 72.0 -> 82.1; 78.9 -> 83.7; 92.5 -> 95.7; 91.7 -> 96.9 | Helped |
| Regularize the decoder instead of the encoder (ablation) | geometry-preserving regularization on Shapes3D · Shapes3D | DCI-D: 70.9 -> 82.1; 79.6 -> 83.7; 92.5 -> 95.7; 93.5 -> 96.9 | Helped |
| Replace decoder distortion regularization with decoder GMG (ablation) | cosine-cost geometry regularization · Shapes3D | DCI-D: 76.8 -> 82.1; 81.3 -> 83.7; 87.5 -> 95.7; 91.9 -> 96.9 | Helped |
| Add HFS (ablation) | β-VAE/β-TCVAE-based models · SmallNORB and Cars3D | DCI-D | No clear effect |
| Use Jacobian regularization instead of GMG (ablation) | β-VAE on Shapes3D · Shapes3D | DCI-D: 61.4 ± 12.8 vs 82.1 ± 4.5 | Hurt |
Surprising findings
- Angle preservation (cosine cost) consistently outperforms scaled-distance preservation for disentanglement, despite distances being the more common choice in prior work.
- Regularizing the decoder works better than regularizing the encoder.
Limitations
- Gains are modest or within standard deviation on some dataset and model combinations (e.g., SmallNORB, Cars3D with HFS).
- Weak convexity constants can grow with the reference distribution support and sample geometry, making optimization harder for distance costs.
- Evaluated only on small 64x64 synthetic-factor image benchmarks with DCI-D as the main metric.
Experimental setup
Datasets: Shapes3D DSprites SmallNORB Cars3D
Benchmarks: Shapes3D DSprites SmallNORB Cars3D
Metrics: DCI-D gradient alignment (stability)
Baselines: beta-VAE beta-TCVAE HFS Jacobian regularization (Jac) distortion regularizer (DST)
Builds on: beta-VAE beta-TCVAE HFS Monge gap distortion regularization (Lee et al. 2023 / WAE-based isometry)
Closest papers by meaning
- On the Transfer of Disentangled Representations in Realistic Settings (ICLR 2021 · similarity 0.89)
- Orthogonality-Enforced Latent Space in Autoencoders: An Approach to Learning Disentangled Representations (ICML 2023 · similarity 0.88)
- Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations (ICML 2019 · similarity 0.88)
- Disentangled Representation Learning via Flow Matching (NeurIPS 2026 · similarity 0.87)
- Disentanglement Learning via Topology (ICML 2024 · similarity 0.87)
- Enriching Disentanglement: From Logical Definitions to Quantitative Metrics (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.