Research map / Graph learning / Graph self-supervised learning
Visual Distant Supervision for Scene Graph Generation
Yuan Yao, Ao Zhang, Xu Han, Mengdi Li, Cornelius Weber, Zhiyuan Liu, Stefan Wermter, Maosong Sun
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add denoising (DNS) plus external CLIP signal to distant supervision (ablation) | Motif trained on distant labels · Visual Genome (20 predicates) | PredCls R@50: 50.23 -> 53.40 | Improved |
| add external CLIP semantic signal to raw distant labels (ablation) | Raw Label · Visual Genome (20 predicates) | PredCls R@50: 30.61 -> 38.21 | Improved |
| train a scene graph model (Motif) on raw distant labels instead of using labels directly (ablation) | Raw Label · Visual Genome (20 predicates) | PredCls R@50: 30.61 -> 50.23 | Improved |
| use the external CLIP signal alone as predictor (ablation) | EXT · Visual Genome (20 predicates) | PredCls R@50: 6.64 | Hurt |
| add Cleanness Loss denoising baseline to distantly supervised Motif (ablation) | Motif trained on distant labels · Visual Genome (20 predicates) | PredCls R@50: 50.23 -> 51.10 | Improved |
| add Cleanness Loss denoising baseline to distantly supervised Motif (ablation) | Motif trained on distant labels · Visual Genome (20 predicates) | SGCls R@50: 24.90 -> 24.06 | Hurt |
| pre-train on distantly labeled data before fine-tuning on human labels | Motif (fully supervised) · Visual Genome (20 predicates) | PredCls R@50: 67.93 -> 73.22 | Improved |
| add the denoising framework to semi-supervised pretraining (ablation) | Motif + Pretrain · Visual Genome (20 predicates) | PredCls R@50: 73.22 -> 76.28 | Improved |
| add the denoising framework to semi-supervised pretraining (ablation) | Motif + Pretrain · Visual Genome (20 predicates) | PredCls mR@50: 60.44 -> 60.20 | No clear effect |
| use distantly supervised model without human labels instead of weak supervision | Weak Supervision · Visual Genome (20 predicates) | PredCls R@50: 44.96 -> 53.40 | Improved |
| swap general knowledge base for ideal knowledge base from Visual Genome annotations | Motif + DNS + EXT · Visual Genome (20 predicates, ideal KB) | PredCls mR@50: 37.68 -> 50.87 | Improved |
| use semi-supervised distant supervision on the 50-predicate Visual Genome scheme | Motif (fully supervised) · Visual Genome (50 predicates) | PredCls R@50: 63.96 -> 64.43 | Improved |
Closest papers by meaning
- Scene Graph Prediction With Limited Labels (ICCV 2019 · similarity 0.93)
- Scene Graph Generation by Iterative Message Passing (CVPR 2017 · similarity 0.92)
- Not All Relations Are Equal: Mining Informative Labels for Scene Graph Generation (CVPR 2022 · similarity 0.91)
- A Simple Baseline for Weakly-Supervised Scene Graph Generation (ICCV 2021 · similarity 0.91)
- DSGG: Dense Relation Transformer for an End-to-end Scene Graph Generation (CVPR 2024 · similarity 0.90)
- Leveraging Predicate and Triplet Learning for Scene Graph Generation (CVPR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
