Research map / Vision-language and multimodal / Human and embodied modeling
Linking People across Text and Images Based on Social Relation Reasoning
Yang Lei, Peizhi Zhao, Pijian Li, Yi Cai, Qingbao Huang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add social-relation extraction and cross-modal reasoning | SRR · Who's Waldo test | Acc@0.5: 68.79 ±0.97 -> 74.49 ±1.10 | Improved |
| Add social-relation extraction and cross-modal reasoning | SRR · FL:MSRE test | Acc@0.5: 66.07 ±1.05 -> 76.26 ±1.11 | Improved |
| Add social-relation extraction and cross-modal reasoning | SRR trained on Who's Waldo · FL:MSRE test (cross-dataset) | Acc@0.5: 61.05 -> 60.13 | Hurt |
| Use a pretrained Transformer SRE encoder (ablation) | Transformer SRE · Who's Waldo val | Acc@0.5: 63.4 -> 74.5 | Improved |
| Add GCN G optimization (ablation) | Transformer SRE · Who's Waldo val | Acc@0.5: 70.1 -> 72.3 | Improved |
| Add GCN S optimization alongside GCN G (ablation) | Transformer SRE · Who's Waldo val | Acc@0.5: 72.3 -> 74.5 | Improved |
| Use a pretrained LSTM SRE encoder (ablation) | LSTM SRE · Who's Waldo val | Acc@0.5: 64.2 -> 71.2 | Improved |
| Add GCN G and GCN S optimization (ablation) | LSTM SRE · Who's Waldo val | Acc@0.5: 70.7 -> 71.2 | Improved |
| Add GCN S optimization alongside GCN G (ablation) | LSTM SRE · Who's Waldo val | Acc@0.5: 69.5 -> 71.2 | Improved |
| Use a pretrained BiLSTM SRE encoder (ablation) | BiLSTM SRE · Who's Waldo val | Acc@0.5: 65.1 -> 71.1 | Improved |
| Add GCN G and GCN S optimization (ablation) | BiLSTM SRE · Who's Waldo val | Acc@0.5: 69.1 -> 71.1 | Improved |
| Add GCN S optimization alongside GCN G (ablation) | BiLSTM SRE · Who's Waldo val | Acc@0.5: 69.5 -> 71.1 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 2 on most shared results.
- Pseudo-Q — Pseudo-Q: Generating Pseudo Language Queries for Visual Grounding (CVPR 2022) · better on 0 of 1 shared result
- TransVG — TransVG: End-to-End Visual Grounding With Transformers (ICCV 2021) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- FL-MSRE: A Few-Shot Learning based Approach to Multimodal Social Relation Extraction (AAAI 2021 · similarity 0.88)
- SocialGPT: Prompting LLMs for Social Relation Reasoning via Greedy Segment Optimization (NeurIPS 2024 · similarity 0.88)
- Cross-Modal Implicit Relation Reasoning and Aligning for Text-to-Image Person Retrieval (CVPR 2023 · similarity 0.88)
- Graph-Based Social Relation Reasoning (ECCV 2020 · similarity 0.87)
- Relationship-Embedded Representation Learning for Grounding Referring Expressions (TPAMI 2020 · similarity 0.87)
- Visual Social Relationship Recognition (IJCV 2020 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
