Research map / Image generation / Editing and style transfer
Concept Lancet: Image Editing with Compositional Representation Transplant
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Concept Lancet (CoLan), a zero-shot plug-and-play method that sparsely decomposes diffusion latents (text embedding or score) over a VLM-selected concept dictionary built from a new 150K-stimulus dataset, then transplants target concepts to set edit strength per image.
Problem
Diffusion-based image editing requires choosing an edit magnitude along a concept direction; too little fails the edit and too much harms consistency, and the right strength varies per source image.
Approach
Decompose the source latent (text embedding or score) as a sparse linear combination of visual concept vectors estimated from curated stimuli, then replace (or add/remove) the source concept atom with the target atom, so the coefficients implicitly set edit strength per image.
Key results
- On the P2P-Zero backbone, CoLan lowers StruDist and LPIPS by nearly 50% and raises PSNR and SSIM by about 10%.
- InfEdit with CoLan in the score space gives the most outstanding performance across multiple metrics on PIE-Bench.
- Sparse decomposition takes 0.153 s out of 38.74 s editing time for P2P-Zero (0.394%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace VecAdd with CoLan concept transplant | P2P-Zero · PIE-Bench | StruDist: 53.04 -> 15.91 | Helped |
| Replace VecAdd with CoLan concept transplant | P2P-Zero · PIE-Bench | PSNR: 17.65 -> 23.08 | Helped |
| Replace VecAdd with CoLan concept transplant | P2P-Zero · PIE-Bench | LPIPS: 273.8 -> 120.3 | Helped |
| Replace VecAdd with CoLan concept transplant | P2P-Zero · PIE-Bench | Target Concept: 20.81 -> 21.13 | Helped |
| Replace VecAdd with CoLan concept transplant | P2P-Zero · PIE-Bench | SSIM: 61.78 -> 75.82; 76.60 -> 83.55 | Mixed |
| Increase dictionary size from N = 5 to N = 30 (ablation) | CoLan with P2P-Zero · PIE-Bench | LPIPS: 135.6 -> 72.85 | Helped |
| Increase dictionary size from N = 5 to N = 30 (ablation) | CoLan with P2P-Zero · PIE-Bench | Target Concept: 20.83 -> 21.14 | Helped |
| Increase dictionary size from N = 5 to N = 30 (ablation) | CoLan with InfEdit (S) · PIE-Bench | LPIPS: 56.28 -> 53.11 | Helped |
| Increase dictionary size from N = 5 to N = 30 (ablation) | CoLan with InfEdit (S) · PIE-Bench | Target Concept: 22.05 -> 22.40 | Helped |
Surprising findings
- DI with P2P achieves the best StruDist, slightly better than CoLan, which the authors attribute to StruDist's dependence on a specific DINO-V2 transformer.
Limitations
- Relies on a VLM to parse image-prompt tuples into concepts and on an LLM to synthesize stimuli.
- Evaluation metrics like StruDist are backbone-dependent, so consistency measures are imperfect.
Experimental setup
Datasets: PIE-Bench CoLan-150K
Benchmarks: PIE-Bench
Metrics: StruDist PSNR LPIPS SSIM CLIP similarity (target image) CLIP similarity (target concept)
Baselines: P2P MasaCtrl P2P-Zero InfEdit Direct Inversion
Builds on: DDIM inversion prompt-to-prompt P2P-Zero InfEdit representation reading
Closest papers by meaning
- Localizing and Editing Knowledge In Text-to-Image Generative Models (ICLR 2024 · similarity 0.89)
- RelationAdapter: Learning and Transferring Visual Relation with Diffusion Transformers (NeurIPS 2025 · similarity 0.88)
- Textualize Visual Prompt for Image Editing via Diffusion Bridge (AAAI 2025 · similarity 0.88)
- An Item Is Worth a Prompt: Versatile Image Editing with Disentangled Control (AAAI 2025 · similarity 0.88)
- The Hidden Language of Diffusion Models (ICLR 2024 · similarity 0.88)
- Semantic Surgery: Zero-Shot Concept Erasure in Diffusion Models (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.