Research map / Graph learning / Knowledge and temporal graphs
Towards Open-Vocabulary Scene Graph Generation with Prompt-Based Finetuning
Tao He, Lianli Gao, Jingkuan Song, Yuan-Fang Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add region-caption pre-training (VRM) instead of prior pretrained model (ablation) | SVRP finetuning · Visual Genome Ov-SGG PredCls | R@50: 42.13 -> 44.73 | Improved |
| use prompt-based finetuning (hard prompt) instead of standard finetuning (ablation) | VRM · Visual Genome Ov-SGG PredCls | R@50: 44.73 -> 46.34 | Improved |
| add soft visual-relation prompt on top of hard prompt (ablation) | VRM · Visual Genome Ov-SGG PredCls | R@50: 46.34 -> 47.62 | Improved |
| add visual-to-textual decoder to soft prompt (ablation) | SVRP · Visual Genome Ov-SGG PredCls | R@50: 46.96 -> 47.62 | Improved |
| add VRM pre-training (ablation) | SVRP · GQA Ov-SGG PredCls | R@50: 31.37 -> 33.08 | Improved |
| add soft prompt instead of standard finetuning (ablation) | VRM · GQA Ov-SGG PredCls | R@50: 33.08 -> 35.26 | Improved |
| use hard prompt instead of standard finetuning (ablation) | VRM · Visual Genome Cs-SGG PredCls | R@50: 51.42 -> 46.83 | Hurt |
| add soft prompt SVRP instead of EBM | SGG model · Visual Genome Ov-SGG PredCls | R@50: 44.09 -> 47.62 | Improved |
| add SVRP instead of EBM | SGG model · Visual Genome Cs-SGG PredCls | R@50: 52.81 -> 54.39 | Improved |
| use SVRP in zero-shot object SGG | SGG model · Visual Genome ZsO-SGG SGCls | R@50: 9.30 | Improved |
| add SVRP instead of EBM | SGG model · Open-Image v6 Ov-SGG | R@50: 52.03 -> 54.45 | Improved |
| add SVRP instead of EBM on fully closed SGG | SGG model · Visual Genome fully closed PredCls | mR@100: 26.0 -> 25.3 | Hurt |
Closest papers by meaning
- Expanding Scene Graph Boundaries: Fully Open-vocabulary Scene Graph Generation via Visual-Concept Alignment and Retention (ECCV 2024 · similarity 0.95)
- Learning To Generate Language-Supervised and Open-Vocabulary Scene Graph Using Pre-Trained Visual-Semantic Space (CVPR 2023 · similarity 0.95)
- From Pixels to Graphs: Open-Vocabulary Scene Graph Generation with Vision-Language Models (CVPR 2024 · similarity 0.94)
- Relation-aware Hierarchical Prompt for Open-vocabulary Scene Graph Generation (AAAI 2025 · similarity 0.93)
- Scene Graph Generation with Role-Playing Large Language Models (NeurIPS 2024 · similarity 0.92)
- Scene Graph-Grounded Image Generation (AAAI 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
