Research map / Image generation / Text-to-image diffusion
FiVA: Fine-grained Visual Attribute Dataset for Text-to-Image Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds FiVA, a ~1M-image synthetic dataset of fine-grained visual attribute pairs generated with Playground-v2.5, and FiVA-Adapter, a Q-Former-based IP-Adapter extension that transfers attributes from one or more reference images to a target subject.
Problem
Text-to-image models struggle to precisely control fine-grained visual attributes (lighting, texture, dynamics, etc.), and existing image-prompt methods mostly capture identity or a coarse "style" and cannot combine attributes from multiple sources.
Approach
Generate attribute-consistent image pairs with a taxonomy-driven prompt pipeline and a hierarchical range-sensitive filter (GPT-4 based), then train a Q-Former-conditioned, attribute-aware image adapter with dual cross-attention supporting multiple reference images.
Key results
- User study: FiVA Sub-Acc 0.817 and Attr&Sub-Acc 0.348, versus best baseline DEADiff 0.605 and 0.260.
- CLIP-Score in-domain 0.228 and out-domain 0.229, the highest among compared methods.
- GPT-4V study: FiVA average Attr&Sub-Acc 0.592, above StyleAligned 0.567 and DB-Lora 0.503.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FiVA-Adapter instead of the strongest baseline, DEADiff | Customized image generation · User Study | Sub-Acc: 0.605 -> 0.817 | Helped |
| Use FiVA-Adapter instead of the strongest baseline, StyleAligned | Customized image generation · User Study | Attr&Sub-Acc: 0.298 -> 0.348 | Helped |
| Use FiVA-Adapter instead of the strongest baseline, DEADiff | Customized image generation · CLIP-Score in-domain | CLIP-Score: 0.211 -> 0.228 | Helped |
| Use FiVA-Adapter instead of the strongest baseline, DEADiff | Customized image generation · CLIP-Score out-domain | CLIP-Score: 0.205 -> 0.229 | Helped |
| Use FiVA-Adapter instead of the strongest baseline, StyleAligned | Customized image generation · GPT Study average | Attr&Sub-Acc: 0.567 -> 0.592 | Helped |
| Add range-sensitive data filtering (ablation) | FiVA-Adapter · Difficult attributes, including lighting | Attribute accuracy | Helped |
| Add range-sensitive data filtering (ablation) | FiVA-Adapter · Image generation quality | Generation quality | Helped |
| Add attribute tag augmentation during Q-former training (ablation) | FiVA-Adapter · Inference with slightly varied attribute wording | Attribute extraction | Helped |
Surprising findings
- Joint attribute-and-subject accuracy stays low for all methods (best 0.348 in user study), which the authors attribute to the difficulty of the new task.
- Design and Rhythm attributes underperform in GPT study, possibly due to small data scale for those attributes.
Limitations
- Dataset depends heavily on the generative model (Playground-v2.5), limiting realism, attribute range and accuracy of pairs.
- Composition techniques and creative photography attributes are hard to generate this way.
- Joint attribute-subject accuracy remains low overall.
Experimental setup
Datasets: FiVA
Benchmarks: FiVA validation set (100 reference images, seven attributes)
Metrics: subject accuracy attribute-and-subject accuracy CLIP score
Baselines: DreamBooth-LoRA IP-Adapter DEADiff StyleAligned
Builds on: IP-Adapter BLIP-Diffusion DEADiff Q-Former
Closest papers by meaning
- Alchemist: Turning Public Text-to-Image Data into Generative Gold (NeurIPS 2025 · similarity 0.87)
- Stylebreeder: Exploring and Democratizing Artistic Styles through Text-to-Image Models (NeurIPS 2024 · similarity 0.86)
- Free-Lunch Color-Texture Disentanglement for Stylized Image Generation (NeurIPS 2025 · similarity 0.86)
- ArtiMuse: Fine-Grained Image Aesthetics Assessment with Joint Scoring and Expert-Level Understanding (CVPR 2026 · similarity 0.85)
- Aesthetic Post-Training Diffusion Models from Generic Preferences with Step-by-step Preference Optimization (CVPR 2025 · similarity 0.85)
- APDDv2: Aesthetics of Paintings and Drawings Dataset with Artist Labeled Scores and Comments (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.