Research map / Image generation / Editing and style transfer
IMPRINT: Generative Object Compositing by Learning Identity-Preserving Representation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IMPRINT, a two-stage diffusion framework for generative object compositing that first pretrains a DINOv2-based object encoder with context-agnostic multi-view object reconstruction, then trains a Stable Diffusion-based compositing generator with the encoder frozen, plus shape-guided mask control.
Problem
Generative object compositing requires preserving object identity while adapting color and geometry to the background; existing methods either lose fine detail or are restricted in pose/view changes.
Approach
Decouple identity preservation from harmonization: first learn a view-invariant, identity-preserving object encoder via multi-view object view synthesis (no background, no camera conditions), then freeze it and train a compositing generator conditioned on its tokens, with multi-level mask shape guidance.
Key results
- On the Pixabay test set, IMPRINT reaches FID 16.45, DINO-score 94.7 and DreamSim 0.083, versus 22.49 FID and 0.142 DreamSim for ObjectStitch.
- In the user study on DreamBooth, IMPRINT is preferred for fidelity over Paint-by-Example, ObjectStitch and TF-ICON at 80.41%, 86.49% and 73.65%, respectively.
- The ablation shows that removing the first-stage ID-preserving pretraining notably degrades compositing quality (Fig. 7).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use IMPRINT rather than ObjectStitch | generative object compositing · Pixabay test set | FID: 22.4934 -> 16.4487 | Helped |
| Use IMPRINT rather than ObjectStitch | generative object compositing · Pixabay test set | DINO-score: 90.3853 -> 94.705 | Helped |
| Fine-tune the CLIP encoder (ablation) | compositing model with CLIP encoder, setting 1 · Pixabay test set | DINO score: 90.385 -> 92.695 | Helped |
| Add video data to training (ablation) | compositing model with fine-tuned CLIP encoder, setting 2 · Pixabay test set | DINO score: 92.695 -> 94.216 | Helped |
| Add an adapter to the DINOv2 encoder (ablation) | compositing model, setting 4 · Pixabay test set | FID: 20.748 -> 20.131 | Helped |
| Add MVImgNet to second-stage training (ablation) | compositing model, setting 6 · Pixabay test set | FID: 17.477 -> 17.947 | Hurt |
| Add MVImgNet to second-stage training (ablation) | compositing model, setting 6 · Pixabay test set | DINO score: 94.164 -> 94.023 | Hurt |
| Add the identity-preserving pretraining stage (ablation) | compositing model without pretraining, setting 7 · Pixabay test set | FID: 17.947 -> 16.449 | Helped |
| Add the identity-preserving pretraining stage (ablation) | compositing model without pretraining, setting 7 · Pixabay test set | DINO score: 94.023 -> 94.705 | Helped |
| Tune the encoder during pretraining (ablation) | two-stage compositing model, setting 8 versus setting 9 · Pixabay test set | FID: 19.286 -> 17.847 | Helped |
| Tune the UNet during pretraining (ablation) | two-stage compositing model, setting 9 versus setting 10 · Pixabay test set | FID: 19.286 -> 16.449 | Helped |
| Increase the temporal sampling window n (ablation) | first-stage object-centric training data · generation quality | generation quality | Hurt |
Surprising findings
- Optimizing the CLIP encoder improves identity capture but reduces variation, and adding too many object-centric MVImgNet pairs degrades generation quality due to missing background information.
Limitations
- Identity preservation drops when the required view change is too large.
- Consistency of small texts or logos may degrade due to latent autoencoder information loss.
Experimental setup
Datasets: MVImgNet Objaverse Pixabay YoutubeVOS VIPSeg PPR10K DreamBooth
Benchmarks: Pixabay test set (1,000 images) DreamBooth test set (113 pairs)
Metrics: FID CLIP-score DINO-score DreamSim user preference
Baselines: Paint-by-Example ObjectStitch TF-ICON AnyDoor
Builds on: Stable Diffusion DINOv2 ObjectStitch Paint-by-Example TF-ICON
Closest papers by meaning
- PAIR Diffusion: A Comprehensive Multimodal Object-Level Image Editor (CVPR 2024 · similarity 0.87)
- Composing Parts for Expressive Object Generation (CVPR 2025 · similarity 0.85)
- Energy-Guided Optimization for Personalized Image Editing with Pretrained Text-to-Image Diffusion Models (AAAI 2025 · similarity 0.85)
- Zero-shot Image Editing with Reference Imitation (NeurIPS 2024 · similarity 0.84)
- Refine-by-Align: Reference-Guided Artifacts Refinement through Semantic Alignment (ICLR 2025 · similarity 0.84)
- Paint by Inpaint: Learning to Add Image Objects by Removing Them First (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.