Research map / Image generation / Editing and style transfer
CaricatureBooth: Data-Free Interactive Caricature Generation in a Photo Booth
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CaricatureBooth, a data-free interactive caricature generator that fine-tunes an ID-preserving SDXL pipeline (InstantID) on TPS-deformed FFHQ faces and uses Bezier-curve landmark dragging at inference.
Problem
Caricature generation lacks large-scale caricature data and existing methods either need paired caricature/photo data or lack intuitive, identity-preserving creative control over exaggeration.
Approach
Synthesize caricature-like training data by applying TPS warps to FFHQ faces and landmarks, train only the IdentityNet of InstantID with line-based landmark conditions, and at inference let users drag Bezier-curve control points on fitted facial landmarks to drive exaggerated deformations while keeping identity via IP-Adapter/ArcFace embeddings.
Key results
- Ours achieves CLIP-T 31.19, Reward 0.57, PickScore 21.74 versus best baseline DemoCaricature at 25.54, 0.38, 20.80 in Table 1.
- User study scores: ID 0.52 and overall 0.87 for the proposed method versus 0.11 and 0.07 for DemoCaricature.
- Generates a caricature in under 10 seconds and is reported over 5x faster than DemoCaricature at inference.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TPS-deformed face training data and line-based landmark conditioning with identity-preserving diffusion | CaricatureBooth · Caricature test set | CLIP-T: 25.54 -> 31.19 | Helped |
| Use TPS-deformed face training data and line-based landmark conditioning with identity-preserving diffusion | CaricatureBooth · Caricature test set | ID: 0.11 -> 0.52 | Helped |
| Use TPS-deformed face training data and line-based landmark conditioning with identity-preserving diffusion | CaricatureBooth · Caricature test set | Overall: 0.07 -> 0.87 | Helped |
| Use TPS-deformed face training data and line-based landmark conditioning instead of five-point landmark conditioning | CaricatureBooth · Exaggerated landmark conditions | shape control | Helped |
Limitations
- Deformation flexibility depends heavily on the TPS preprocessing, which may bias results versus artist-made morphing.
- Highly exaggerated landmark conditions reduce identity consistency, especially for full-body cartoon images.
- ID consistency could not be measured reliably with automatic face recognition on exaggerated faces, so relied on a user study.
Experimental setup
Datasets: FFHQ
Benchmarks: CaricatureBooth 20-identity 20-layout test set
Metrics: CLIP-T ImageReward PickScore ID similarity (user study) overall user satisfaction (user study)
Baselines: StyleCariGAN CariGANs WarpGAN DemoCaricature InstantID InstantID with MultiControl
Builds on: InstantID ControlNet IP-Adapter DemoCaricature Stable Diffusion XL
Closest papers by meaning
- DemoCaricature: Democratising Caricature Generation with a Rough Sketch (CVPR 2024 · similarity 0.88)
- G2L-CariGAN: Caricature Generation from Global Structure to Local Features (AAAI 2024 · similarity 0.85)
- MagicPose: Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusion (ICML 2024 · similarity 0.84)
- Generative Photomontage (CVPR 2025 · similarity 0.83)
- PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalization (CVPR 2024 · similarity 0.83)
- Instant3dit: Multiview Inpainting for Fast Editing of 3D Objects (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.