Research map / Image generation / Editing and style transfer
Visual Representation Learning through Causal Intervention for Controllable Image Editing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CIDiffuser, a diffusion-based causal visual representation learning framework that uses a direct causal effect learning module and SCM to remove spurious attribute correlations for controllable image editing.
Problem
Controllable image editing suffers from spurious correlations between semantically dependent visual attributes, so edits to one attribute unintentionally change unrelated attributes.
Approach
Decomposes image representation into causal semantic representations (via an SCM with a learned causal effect matrix estimated by direct causal effect learning with interventions and an influence-function reweighted loss) and low-level stochastic representations, decoded by a conditional DDIM to reconstruct and edit images.
Key results
- On CelebA-smile, CIDiffuser reaches FID 56.7, IS 3.053, KID 0.027, versus DiffAE FID 51.6 (better FID for DiffAE).
- On CelebA-age MIC, CIDiffuser scores 0.944 vs 0.944 for SCM-VAE; on Pendulum MIC 0.981 vs CFI-VAE 0.921 and CausalDiffAE 0.911.
- Ablation on CelebA-smile shows removing the supervised loss Ls drops LQ from 0.994 to 0.844 and TAD from 0.481 to 0.039.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use CIDiffuser instead of CausalVAE | CelebA-smile representations · CelebA-smile | MIC: 0.837 -> 0.890 | Helped |
| use CIDiffuser instead of SCM-VAE | CelebA-age representations · CelebA-age | TIC: 0.889 -> 0.903 | Helped |
| use CIDiffuser instead of CFI-VAE | Pendulum representations · Pendulum | TIC: 0.981 -> 0.865 | Hurt |
| use CIDiffuser instead of DiffAE | image generation · CelebA | FID: 51.6 -> 56.7 | Hurt |
| use CIDiffuser instead of DiffAE | image generation · CelebA | KID: 0.041 -> 0.027 | Helped |
| add causal modeling (CM) (ablation) | CIDiffuser · CelebA-smile | Attr: 4 -> 6 | Helped |
| add causal modeling (CM) (ablation) | CIDiffuser · CelebA-smile | FID: 76.7 -> 56.7 | Helped |
| add supervised loss Ls (ablation) | CIDiffuser · CelebA-smile | LQ: 0.844 -> 0.994 | Helped |
| add supervised loss Ls (ablation) | CIDiffuser · CelebA-smile | Attr: 2 -> 6 | Helped |
| add supervised loss Ls (ablation) | CIDiffuser · CelebA-smile | FID: 89.5 -> 56.7 | Helped |
| add causal effect loss Ld (ablation) | CIDiffuser · CelebA-smile | TAD: 0.459 -> 0.481 | Helped |
| add causal effect loss Ld (ablation) | CIDiffuser · CelebA-smile | LQ: 0.882 -> 0.994 | Helped |
Surprising findings
- DiffAE achieves better quantitative image quality (FID 51.6) than CIDiffuser (FID 56.7), which the authors acknowledge.
- Removing the causal modeling module increases LQ to 0.999 but reduces the number of captured attributes from 6 to 4.
Limitations
- Experiments limited to two datasets (CelebA subsets and a synthetic Pendulum dataset).
- Image quality metrics (FID) lag behind DiffAE on CelebA.
- Batch size of 16 and 50 training epochs due to compute constraints.
Experimental setup
Datasets: CelebA Pendulum
Benchmarks: CelebA Pendulum
Metrics: MIC TIC TAD Attr FID IS KID AUROC
Baselines: Beta-VAE ConditionalVAE DiffAE Infodiff CausalVAE DEAR SCM-VAE CFI-VAE CausalDiffAE PDAE DBAE CausalGAN
Builds on: CausalVAE DEAR CFI-VAE DiffAE InfoDiffusion
Closest papers by meaning
- Localizing and Editing Knowledge In Text-to-Image Generative Models (ICLR 2024 · similarity 0.89)
- Causal Inference over Visual-Semantic-Aligned Graph for Image Classification (AAAI 2025 · similarity 0.88)
- From Prompts to Tokens: Internalizing Causal Supervision in Vision-Language Model for Multi-Image Causal Reasoning (ICML 2026 · similarity 0.88)
- Concept Lancet: Image Editing with Compositional Representation Transplant (CVPR 2025 · similarity 0.87)
- Towards Safe Concept Transfer of Multi-Modal Diffusion via Causal Representation Editing (NeurIPS 2024 · similarity 0.87)
- CausalVerse: Benchmarking Causal Representation Learning with Configurable High-Fidelity Simulations (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.