Research map / Vision-language and multimodal / Multimodal LLMs
ROVI: A VLM-LLM Re-Captioned Dataset for Open-Vocabulary Instance-Grounded Text-to-Image Generation
Cihang Peng, Qiming Hou, Zhong Ren, Kun Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pre-detection VLM-LLM re-captioning instead of web caption to OVD (ablation) | ROVI pipeline · Data generation methods (5K images) | distinct categories: 3001 -> 25374 | Improved |
| pre-detection VLM-LLM re-captioning instead of VLM description sent directly to OVD (ablation) | ROVI pipeline · Data generation methods (5K images) | avg boxes per image: 9.20 -> 24.01 | Improved |
| pre-detection VLM-LLM re-captioning instead of RAM tagging then OVD (ablation) | ROVI pipeline · Data generation methods (5K images) | avg boxes per image: 11.54 -> 24.01 | Improved |
| replace RAM with RAM++ tagging (ablation) | InstanceDiffusion-style pipeline · Data generation methods (5K images) | avg boxes per image: 11.54 -> 11.58 | No clear effect |
| fuse four OVDs instead of a single detector (ablation) | ROVI detection stage · 5K sampled images | unique category share: 17.0%, 11.9%, 15.8%, 9.4% unique categories per OVD | Improved |
| train GLIGEN on ROVI instead of official data | GLIGEN (SD v1.4) · ROVI validation set | FID: 19.3 -> 15.7 | Improved |
| train GLIGEN on ROVI instead of official data | GLIGEN (SD v1.4) · ROVI validation set | aesthetic score: 5.33 -> 5.83 | Improved |
| train GLIGEN on ROVI instead of official data | GLIGEN (SD v1.4) · ROVI validation set | instance pass rate: 0.750 -> 0.880 | Improved |
| train GLIGEN on ROVI instead of official data | GLIGEN (SD v1.4) · Open Images validation set | instance pass rate: 0.812 -> 0.872 | Improved |
| fine-tune with instance grounding on ROVI vs base SD v1.4 | GLIGEN (SD v1.4) · ROVI validation set | CLIP similarity: 0.283 -> 0.286 | No clear effect |
| GLIGEN on ROVI vs InstanceDiffusion in user study | GLIGEN (SD v1.4) · User study (200 inputs) | aesthetic win rate: 83.3% | Improved |
| GLIGEN on ROVI vs MIGC in user study | GLIGEN (SD v1.4) · User study (200 inputs) | instance alignment win rate: 94.8% | Improved |
Closest papers by meaning
- Hyperbolic Learning with Synthetic Captions for Open-World Detection (CVPR 2024 · similarity 0.87)
- LLMDet: Learning Strong Open-Vocabulary Object Detectors under the Supervision of Large Language Models (CVPR 2025 · similarity 0.87)
- DetCLIPv3: Towards Versatile Generative Open-vocabulary Object Detection (CVPR 2024 · similarity 0.87)
- A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions (CVPR 2024 · similarity 0.87)
- Leveraging Unpaired Data for Vision-Language Generative Models via Cycle Consistency (ICLR 2024 · similarity 0.87)
- Text-Printed Image: Bridging the Image-Text Modality Gap for Text-centric Training of Large Vision-Language Models (CVPR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
