Research map / Image generation / Text-to-image diffusion
Learning Disentangled Identifiers for Action-Customized Text-to-Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Action-Disentangled Identifier (ADI), an inversion method that learns action-specific identifiers from exemplar images on Stable Diffusion via layer-wise identifier tokens and gradient masking, and introduces the ActionBench benchmark for action customization.
Problem
Subject-driven customization methods learn appearance rather than actions and entangle action with context (e.g., human appearance), so they fail to generate specified actions for new subjects, especially animals.
Approach
Expands textual conditioning with layer-wise identifier tokens and masks gradient updates on action-agnostic channels, using gradient differences over context-different and action-different sample pairs to decide which channels to mask.
Key results
- ADI achieves 51.09% total accuracy on ActionBench, versus 27.17% for Stable Diffusion and 20.92% for P+ (human evaluation).
- ADI reaches 60.33% action accuracy, far above Custom Diffusion (29.62%) and P+ (26.90%).
- Ablations show removing layer-wise tokens or either gradient mask degrades action inversion or subject appearance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ADI instead of Stable Diffusion | text-to-image action customization · ActionBench | total accuracy (%): 27.17 -> 51.09 | Helped |
| Use ADI instead of the strongest baseline, Stable Diffusion | text-to-image action customization · ActionBench | action accuracy (%): 30.71 -> 60.33 | Helped |
| Use ADI instead of the strongest baseline, DreamBooth | text-to-image action customization · ActionBench | subject accuracy (%): 95.65 -> 85.87 | Hurt |
| Add layer-wise identifier tokens (ablation) | ADI · ActionBench | action customization quality | Helped |
| Add gradient masking learned from context-different pairs (ablation) | ADI · ActionBench | action customization quality | Helped |
| Add gradient masking learned from action-different pairs (ablation) | ADI · ActionBench | action customization quality | Helped |
| Preserve the intended masked-gradient channels instead of reversing the mask (ablation) | ADI · ActionBench | action customization quality | Helped |
| Use intersection rather than union to merge gradient masks (ablation) | ADI · ActionBench | action customization quality | Helped |
| Use masking ratio β close to 0.5 (ablation) | ADI · ActionBench | action customization quality | Helped |
| Replace the learned gradient mask with uniform channel masking (ablation) | ADI · ActionBench | action customization quality | Hurt |
| Replace the learned gradient mask with random channel masking (ablation) | ADI · ActionBench | action customization quality | Hurt |
| Mask channels with the lowest gradient values instead of learning the mask (ablation) | ADI · ActionBench | action customization quality | Hurt |
Surprising findings
- Standard subject-driven methods such as DreamBooth, Textual Inversion and ReVersion reach near-zero total accuracy (about 2%) because they fail to capture actions at all.
- Stable Diffusion with text-only action prompts achieves higher total accuracy than all subject-driven baselines, indicating they do not exploit exemplar images well.
Limitations
- Relies on DreamBooth-generated action-different samples, which may be inconsistent with action descriptions.
- Optimal masking ratio beta may vary across actions and must be tuned.
- Evaluation covers only eight actions and 23 subjects, judged by human evaluators.
Experimental setup
Datasets: ActionBench
Benchmarks: ActionBench
Metrics: action accuracy subject accuracy total accuracy
Baselines: Stable Diffusion ControlNet DreamBooth Textual Inversion ReVersion Custom Diffusion P+
Builds on: Textual Inversion DreamBooth P+ Stable Diffusion Custom Diffusion
Closest papers by meaning
- Action Imitation in Common Action Space for Customized Action Image Synthesis (NeurIPS 2024 · similarity 0.88)
- Prompt-guided Disentangled Representation for Action Recognition (NeurIPS 2025 · similarity 0.85)
- Decoupled Textual Embeddings for Customized Image Generation (AAAI 2024 · similarity 0.85)
- DynASyn: Multi-Subject Personalization Enabling Dynamic Action Synthesis (AAAI 2025 · similarity 0.85)
- Kronecker Mask and Interpretive Prompts are Language-Action Video Learners (ICLR 2025 · similarity 0.84)
- DVANet: Disentangling View and Action Features for Multi-View Action Recognition (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.