Research map / Image generation / Text-to-image diffusion
CustAny: Customizing Anything from A Single Example
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds the MC-IDC dataset (315k reference-target pairs across 10k categories) and proposes CustAny, a zero-shot text-to-image customization framework for general objects using DINOv2+MAE ID features, dual-level injection, and an ID-aware decoupling loss.
Problem
Zero-shot customization of arbitrary objects from a single reference image and text prompt while preserving identity and text editability; existing object-agnostic methods are limited to specialized domains such as faces or garments, and no large general-object ID dataset exists.
Approach
Combine complementary self-supervised features (DINOv2 for detail, MAE for color/structure) for ID extraction, inject ID globally by fusing class tokens into the class-word text embedding and locally via patch-token cross-attention, and use a decoupling branch with a masked CLIP image feature plus contrastive loss to separate ID from non-ID attributes like pose.
Key results
- On general objects, CustAny reaches FID 47.09 and DINO-i 65.13 versus IP-Adapter's 70.32 and 44.94.
- On human customization, CustAny gets FaceSim 78.54 versus 65.51 for IP-Adapter and 64.10 for PhotoMaker.
- Ablation: combining DINOv2 and MAE features improves CLIP-i to 82.16 and DINO-i to 65.13 over single-model extractors.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CustAny instead of IP-Adapter | General object customization · General objects | DINO-i: 44.94 -> 65.13 | Helped |
| Use CustAny instead of IP-Adapter | Human customization · Human customization | FaceSim: 65.51 -> 78.54 | Helped |
| Use CustAny instead of IP-Adapter | Virtual try-on · Virtual try-on | CLIP-t: 25.03 -> 22.42 | Hurt |
| Use CustAny instead of IP-Adapter | Virtual try-on · Virtual try-on | FID: 104.47 -> 50.65 | Helped |
| Combine DINOv2 and MAE features instead of using DINOv2 alone (ablation) | CustAny ID extraction · General objects | DINO-i: 63.71 -> 65.13 | Helped |
| Combine DINOv2 and MAE features instead of using MAE alone (ablation) | CustAny ID extraction · General objects | DINO-i: 59.79 -> 65.13 | Helped |
| Combine DINOv2 and MAE features instead of using CLIP alone (ablation) | CustAny ID extraction · General objects | DINO-i: 59.45 -> 65.13 | Helped |
| Use dual-level ID injection instead of global-only injection (ablation) | CustAny · General objects | CLIP-i: 78.76 -> 82.16 | Helped |
| Use dual-level ID injection instead of local-only injection (ablation) | CustAny · General objects | CLIP-i: 81.24 -> 82.16 | Helped |
| Add ID-aware decoupling (ablation) | CustAny · General objects | CLIP-i: 81.86 -> 82.16 | Helped |
Limitations
- Evaluation relies on a self-built 1,000-sample set rather than a public benchmark.
- Built on SD1.5, an older backbone.
Experimental setup
Datasets: MC-IDC web-crawled images movie frames publicly available datasets
Benchmarks: 1,000-sample internal evaluation set (general, human, virtual try-on)
Metrics: FID CLIP-i CLIP-t DINO-i FaceSim DiverSim-i
Baselines: IP-Adapter PhotoMaker InstantID MagicClothing
Builds on: Stable Diffusion IP-Adapter PhotoMaker InstantID DisenBooth
Closest papers by meaning
- AnyDoor: Zero-shot Object-level Image Customization (CVPR 2024 · similarity 0.84)
- Orthogonal Adaptation for Modular Customization of Diffusion Models (CVPR 2024 · similarity 0.84)
- PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding (CVPR 2024 · similarity 0.84)
- DreamRelation: Bridging Customization and Relation Generation (CVPR 2025 · similarity 0.83)
- ID-Patch: Robust ID Association for Group Photo Personalization (CVPR 2025 · similarity 0.83)
- DisEnvisioner: Disentangled and Enriched Visual Prompt for Customized Image Generation (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.