Research map / Probabilistic and scientific modeling / Causal discovery
CausalVerse: Benchmarking Causal Representation Learning with Configurable High-Fidelity Simulations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CausalVerse, a high-fidelity simulated visual benchmark (about 200k images and video frames across 24 scenes in four domains) with access to ground-truth causal structures, and benchmarks representative CRL methods on it.
Problem
Evaluating causal representation learning (CRL) requires ground-truth causal variables and structures, which real-world data lacks, while simple synthetic benchmarks lack realism.
Approach
A hierarchical domain-scene-instance benchmark rendered with Blender, Unreal Engine 4, Bullet, Robosuite and Habitat, where each scene has a predefined causal graph and the full simulation process is exposed so users can configure assumptions such as domain labels, temporal dependencies, and intervention histories.
Key results
- Supervised upper bound reaches average MCC 0.9916 and R^2 0.9891 on image scenes, while the best unsupervised method (Sufficient Change) averages MCC 0.5768.
- On video scenes, all temporal CRL methods (IDOL, CaRiNG, TDRL, TCL, iVAE) reach MCC of only about 0.17 to 0.25.
- Sufficient Change with one-scene or wrong scene labels degrades; wrong labels yield an average R^2 of -2.07 in the ablation table.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply Sufficient Change rather than Mechanism Sparsity | Causal representation learning on the three static scenes · Ball on the Slope, Cylinder Spring, Light Refraction (Avg) | MCC: 0.2560 -> 0.5768 | Helped |
| Use supervised training with ground-truth latents rather than Sufficient Change | Causal representation learning on the three static scenes · Ball on the Slope, Cylinder Spring, Light Refraction (Avg) | MCC: 0.5768 -> 0.9916 | Helped |
| Reduce the number of informative scenes from Four-Scenes to One-Scene (ablation) | Sufficient Change · Static image generation | MCC: 0.3264 -> 0.3197 | Hurt |
| Reduce the number of informative scenes from Four-Scenes to One-Scene (ablation) | Sufficient Change · Static image generation | R^2: 0.2898 -> 0.2671 | Hurt |
| Corrupt the scene labels (ablation) | Sufficient Change · Static image generation | MCC: 0.3264 -> 0.1412 | Hurt |
| Corrupt the scene labels (ablation) | Sufficient Change · Static image generation | R^2: 0.2898 -> $-6.7706$ | Hurt |
| Generate enough views to satisfy the sufficient-change assumption | Sufficient Change on Cylinder Spring · Cylinder Spring | MCC | Helped |
| Evaluate on Robotics Study rather than Fall Simple | Temporal causal representation learning methods · Fall Simple vs Robotics Study | R^2 | Helped |
Surprising findings
- Wrong domain labels are more harmful than reducing the number of domains, sometimes producing large negative R^2 values.
- Contrastive Learning reaches R^2 above 0.96 on image scenes despite low MCC, consistent with block-wise rather than component-wise identifiability.
Limitations
- Causal structures are handcrafted and may not capture real-world complexity or ambiguity.
- Lacks natural noise such as sensor variability; sim-to-real gap remains.
- Most CRL methods were evaluated under assumptions they do not satisfy, so results reflect assumption violations as much as method quality.
Experimental setup
Datasets: CausalVerse
Benchmarks: CausalVerse
Metrics: MCC R^2 over-complete MCC
Baselines: Sufficient Change Mechanism Sparsity Multiview Contrastive Learning IDOL CaRiNG TDRL TCL iVAE
Builds on: Causal3DIdent iVAE TCL IDOL CaRiNG
Closest papers by meaning
- Sanity Checking Causal Representation Learning on a Simple Real-World System (ICML 2025 · similarity 0.92)
- Causal Representation Learning Made Identifiable by Grouping of Observational Variables (ICML 2024 · similarity 0.89)
- Unifying Causal Representation Learning with the Invariance Principle (ICLR 2025 · similarity 0.88)
- From Prompts to Tokens: Internalizing Causal Supervision in Vision-Language Model for Multi-Image Causal Reasoning (ICML 2026 · similarity 0.88)
- CaRiNG: Learning Temporal Causal Representation under Non-Invertible Generation Process (ICML 2024 · similarity 0.88)
- Sim-to-Real Causal Transfer: A Metric Learning Approach to Causally-Aware Interaction Representations (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.