Research map / Image generation / Editing and style transfer
Pattern Analogies: Learning to Perform Programmatic Image Edits by Analogy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces a pattern-analogy approach for programmatic image edits, combining a pattern DSL (SPLITWEAVE) for synthetic analogical quartets with TRIFUSER, a latent diffusion model conditioned on fused multi-encoder features and 3D positional encodings.
Problem
Editing structured pattern images programmatically (changing the underlying generating rules) is hard; visual program inference struggles with complex patterns and yields unwieldy programs, and existing analogical image editors focus on appearance edits.
Approach
Specify edits with a simple analogy pair (A, A') and apply them to a complex target B via a diffusion model trained on synthetic quartets generated by consistently editing DSL programs for A and B; the model adds multi-level feature fusion from CLIP-like and DINOv2 encoders plus 3D positional encodings identifying source image and location.
Key results
- In a two-alternative forced-choice study with 42 participants, TriFuser was preferred over Analogist (89.24%), LatentMod (80.59%), and Inpainter (72.05%).
- On the synthetic validation set, TriFuser achieves DSIM 0.074 and LPIPS 0.304, better than Inpainter (0.092 and 0.371) on perceptual metrics, though Inpainter has slightly higher SSIM (0.713 vs 0.704).
- Ablation shows removing 3D positional encoding degrades DSIM from 0.074 to 0.147; the unmodified Image Variation base model reaches DSIM 0.585.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use TRIFUSER instead of Analogist | analogical pattern editing · Adobe Stock pattern test set | preference rate: 89.24% | Helped |
| use TRIFUSER instead of LatentMod | analogical pattern editing · Adobe Stock pattern test set | preference rate: 80.59% | Helped |
| use TRIFUSER instead of Inpainter | analogical pattern editing · Adobe Stock pattern test set | preference rate: 72.05% | Helped |
| add 3D positional encoding (ablation) | TRIFUSER · Synthetic validation set | DSim: 0.147 -> 0.074 | Helped |
| add lower-level features (ablation) | TRIFUSER · Synthetic validation set | DSim: 0.087 -> 0.074 | Helped |
| add feature mixing (ablation) | TRIFUSER · Synthetic validation set | DSim: 0.098 -> 0.074 | Helped |
| add TRIFUSER modifications rather than using the unmodified Image Variation model (ablation) | Image Variation [54] · Synthetic validation set | DSim: 0.585 -> 0.074 | Helped |
Surprising findings
- Inpainter achieves slightly higher SSIM than TriFuser on the synthetic validation set, though TriFuser wins on other perceptual metrics.
Limitations
- Training covers only two pattern styles (MTP and SFP); other styles are evaluated only qualitatively for generalization.
- Each method's output is best-of-k selected (k=9 for human study, k=5 for synthetic metrics), so reported results depend on selection.
Experimental setup
Datasets: SPLITWEAVE synthetic analogical quartets Adobe Stock pattern test set (50 patterns)
Benchmarks: Adobe Stock pattern test set (50 patterns) synthetic analogical validation set (1000 quartets)
Metrics: DSIM DISTS LPIPS SSIM human preference rate
Baselines: Analogist LatentMod Inpainter (Stable Diffusion inpainting, Image Brush-style) Versatile Diffusion image variation (Base)
Builds on: Versatile Diffusion image variation Image Analogies Structure Mapping Theory LayerDiffuse
Closest papers by meaning
- Text-Driven Fashion Image Editing with Compositional Concept Learning and Counterfactual Abduction (CVPR 2025 · similarity 0.87)
- Concept Lancet: Image Editing with Compositional Representation Transplant (CVPR 2025 · similarity 0.86)
- EditInfinity: Image Editing with Binary-Quantized Generative Models (NeurIPS 2025 · similarity 0.86)
- Zero-shot Image Editing with Reference Imitation (NeurIPS 2024 · similarity 0.86)
- Meta-CoT: Enhancing Granularity and Generalization in Image Editing (CVPR 2026 · similarity 0.86)
- Pathways on the Image Manifold: Image Editing via Video Generation (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.