Research map / Graph learning / Knowledge and temporal graphs
Learning To Generate Scene Graph From Natural Language Supervision
Yiwu Zhong, Jing Shi, Jianwei Yang, Chenliang Xu, Yin Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add weighted loss (ablation) | Ours trained on CC captions · VG SGDet | R@100: 4.1 -> 6.4 | Improved |
| add weighted loss (ablation) | Ours trained on COCO captions · VG SGDet | R@100: 4.5 -> 6.7 | Improved |
| use CC instead of COCO captions (ablation) | Ours with weighted loss · VG SGDet | R@100: 6.4 vs 6.7 | No clear effect |
| combine CC and COCO captions (ablation) | Ours with weighted loss · VG SGDet | R@100: 6.7 -> 7.0 | Improved |
| apply our label assignment to MotifNet (ablation) | MotifNet vs VSPNet · VG SGDet (unlocalized scene graph) | R@100: 7.4 -> 10.7 | Improved |
| use Triplet Transformer instead of MotifNet (ablation) | Ours · VG SGDet (unlocalized scene graph) | R@100: 10.7 -> 11.5 | Improved |
| use OpenImages detector instead of Objects365 (ablation) | Ours · VG SGDet (unlocalized scene graph) | R@100: 6.4 -> 11.5 | Improved |
| add text input to visual input (ablation) | Ours · VG SGDet | R@100: 4.7 -> 11.5 | Improved |
| add visual input to text input (ablation) | Ours · VG SGDet | R@100: 7.7 -> 11.5 | Improved |
| add visual input to text input (ablation) | Ours · VG | detection mAP: 6.9 -> 10.7 | Improved |
| add text input to visual input (ablation) | Ours · VG | detection mAP: 10.6 -> 10.7 | No clear effect |
| use open-set vocabulary of 4273 objects and 677 predicates (ablation) | Ours trained on COCO · VG SGDet | R@100: 4.5 -> 4.8 | Improved |
Closest papers by meaning
- Learning To Generate Language-Supervised and Open-Vocabulary Scene Graph Using Pre-Trained Visual-Semantic Space (CVPR 2023 · similarity 0.93)
- Improving Scene Graph Classification by Exploiting Knowledge from Texts (AAAI 2022 · similarity 0.91)
- Towards Open-Vocabulary Scene Graph Generation with Prompt-Based Finetuning (ECCV 2022 · similarity 0.91)
- Scene Graph Generation with Role-Playing Large Language Models (NeurIPS 2024 · similarity 0.90)
- From Pixels to Graphs: Open-Vocabulary Scene Graph Generation with Vision-Language Models (CVPR 2024 · similarity 0.90)
- DSGG: Dense Relation Transformer for an End-to-end Scene Graph Generation (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
