Research map / Vision-language and multimodal / Human and embodied modeling
FL-MSRE: A Few-Shot Learning based Approach to Multimodal Social Relation Extraction
Hai Wan, Manrong Zhang, Jianfeng Du, Ziling Huang, Yufei Yang, Jeff Z. Pan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add face-image features to the BERT text model, using same-image faces | Proto (BERT) · FC-TF, 3 way 1 shot | accuracy: 40.00±0.11 -> 58.44±0.21 | Improved |
| Add face-image features to the BERT text model, using different-image faces | Proto (BERT) · FC-TF, 3 way 3 shot | accuracy: 51.29±0.16 -> 77.29±0.54 | Improved |
| Add face-image features to the BERT text model, using same-image faces | Proto (BERT) · DRC-TF, 3 way 1 shot | accuracy: 46.28±0.15 -> 54.16±0.22 | Improved |
| Add face-image features to the BERT text model, using different-image faces | Proto (BERT) · DRC-TF, 3 way 3 shot | accuracy: 48.35±0.43 -> 62.30±0.39 | Improved |
| Add face-image features to the BERT text model, using same-image faces | Proto (BERT) · OM-TF, 3 way 1 shot | accuracy: 40.00±0.11 -> 58.44±0.21 | Improved |
| Add face-image features to the BERT text model, using different-image faces | Proto (BERT) · OM-TF, 3 way 3 shot | accuracy: 51.29±0.16 -> 77.29±0.54 | Improved |
| Collect the two faces from different rather than the same image | FL-MSRE · FC-TF, DRC-TF, and OM-TF | accuracy: -0.44±0.10; 0.36±0.12; 0.63±0.15; -0.26±0.22; 3.60±0.22; 5.33±0.39; 0.36±0.66; 0.63±0.25 | Mixed |
| Train on one dataset and test on another, using multimodal faces | FL-MSRE versus Proto (BERT) · DRC-TF trained on OM-TF; OM-TF trained on DRC-TF | accuracy: 38.38±0.16 -> 50.69±0.81; 40.74±0.18 -> 63.83±0.42; 37.27±0.69 -> 55.82±0.76; 44.09±0.69 -> 58.50±0.45 | Improved |
Closest papers by meaning
- Linking People across Text and Images Based on Social Relation Reasoning (AAAI 2023 · similarity 0.88)
- Prototype-Guided Multimodal Relation Extraction based on Entity Attributes (AAAI 2025 · similarity 0.85)
- SocialGPT: Prompting LLMs for Social Relation Reasoning via Greedy Segment Optimization (NeurIPS 2024 · similarity 0.84)
- A Hierarchical Network for Multimodal Document-Level Relation Extraction (AAAI 2024 · similarity 0.83)
- SocialGesture: Delving into Multi-person Gesture Understanding (CVPR 2025 · similarity 0.82)
- Synergistic Anchored Contrastive Pre-training for Few-Shot Relation Extraction (AAAI 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
