Research map / Image generation / Editing and style transfer
Diffusion Counterfactual Generation with Semantic Abduction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes diffusion-based deep structural causal model mechanisms (spatial, semantic, and dynamic abduction with amortised anti-causal guidance) for counterfactual image generation that better preserve identity than VAE/HVAE and DiffSCM baselines.
Problem
Counterfactual image generation must preserve identity, perceptual quality, and faithfulness to a causal model; existing VAE/HVAE and diffusion-based DSCMs trade off these properties and scale poorly.
Approach
Builds diffusion DSCM mechanisms where abduction inverts conditional DDIMs to get spatial noise, adds a variational semantic latent encoded by a CNN for semantic abduction, uses amortised anti-causal classifier-free guidance, and optimises guidance tokens per timestep (counterfactual trajectory alignment) for dynamic abduction.
Key results
- On Morpho-MNIST do(d), the semantic mechanism with omega=1.5 reaches 97.46% effectiveness with composition L1 0.342e-2, versus VCI's 92.48% and 2.05e-2.
- On CelebA-HQ, semantic mechanisms consistently outperform spatial mechanisms in identity preservation (LPIPS) across guidance scales, e.g. 0.077 vs 0.102 for do(g) with null LPIPS-level comparisons at omega=2.
- On EMBED, the guided semantic mechanism removes 95.16% of triangular and 91.69% of circular skin markers in the test set.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use semantic rather than spatial abduction | Guided diffusion counterfactual mechanisms · CelebA-HQ | LPIPS identity preservation: 0.171\,(0.0002) -> 0.096\,(0.0002) | Helped |
| Increase guidance scale (ablation) | Guided diffusion mechanisms · Morpho-MNIST | accuracy under do(d): 99.51 -> 99.90 | Helped |
| Increase guidance scale (ablation) | Guided diffusion mechanisms · Morpho-MNIST | L1 reversibility under do(d): 4.50 -> 5.70 | Hurt |
| Increase p∅ from 0.1 to 0.5 (ablation) | Guided spatial mechanism · Morpho-MNIST do(d) | accuracy: 99.51 -> 93.95 | Hurt |
| Increase p∅ from 0.1 to 0.5 (ablation) | Guided spatial mechanism · Morpho-MNIST null intervention | L1 composition: 2.02 -> 0.660 | Helped |
| Use semantic rather than spatial abduction | Guided counterfactual mechanisms · Morpho-MNIST | identity preservation | Mixed |
| Add counterfactual trajectory alignment with dynamic semantic abduction | Guided semantic mechanism · CelebA-HQ | identity preservation | Helped |
| Add counterfactual trajectory alignment with dynamic semantic abduction | Guided semantic mechanism · CelebA-HQ | effectiveness | Helped |
| Increase semantic mechanism p∅ from 0.1 to 0.2 (ablation) | Guided semantic mechanism · CelebA-HQ | L1 composition: 0.130\,(0.0001) -> 0.122\,(0.0002) | Helped |
| Increase DiffSCM guidance scale and train its unconditional model for 500K rather than 30K steps (ablation) | DiffSCM · Morpho-MNIST do(d) | accuracy: 17.02 -> 98.73 | Helped |
| Increase DiffSCM guidance scale (ablation) | DiffSCM · Morpho-MNIST do(d) | L1 reversibility: 1.31 -> 4.42 | Hurt |
Surprising findings
- Higher guidance scales and smaller classifier-free dropout probability improve effectiveness but degrade identity preservation, a trade-off opposite to the usual L2-level sample-quality focus.
- DiffSCM's effectiveness is poor under its original guidance scale; increasing it recovers effectiveness but worsens reversibility.
Limitations
- Small datasets limit learning of strided trajectories, leading to slower counterfactual inference.
- Causal identifiability and Markovian SCM assumptions may not hold in practice; noisy labels and low resolution can violate the causal graph.
- Unconditional VAE prior makes the semantic mechanism non-identifiable; identifiability left for future work.
Experimental setup
Datasets: Morpho-MNIST CelebA-HQ CelebA EMBED
Benchmarks: Morpho-MNIST CelebA-HQ EMBED
Metrics: composition L1 reversibility L1 effectiveness accuracy MAPE F1 LPIPS identity preservation
Baselines: VAE HVAE VCI DiffSCM
Builds on: DDIM DiffAE DiffSCM Variational Causal Inference (VCI) classifier-free guidance
Closest papers by meaning
- Counterfactual Image Editing (ICML 2024 · similarity 0.90)
- Benchmarking Counterfactual Image Generation (NeurIPS 2024 · similarity 0.89)
- V-CECE: Visual Counterfactual Explanations via Conceptual Edits (NeurIPS 2025 · similarity 0.87)
- Concurrent Image Understanding and Generation: Self-Correcting Coupled Markov Jump Process (NeurIPS 2026 · similarity 0.87)
- Towards Safe Concept Transfer of Multi-Modal Diffusion via Causal Representation Editing (NeurIPS 2024 · similarity 0.87)
- Counterfactual Generative Modeling with Variational Causal Inference (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.