Research map / Vision-language and multimodal / Multimodal LLMs
Ego3S: Select, Strengthen, and Synchronize for Efficient Egocentric Reasoning
Shenshen Li, Kaiyuan Deng, RuoHuai Xie, Xing Xu, Heng Tao Shen, Yazhou Yao, Fumin Shen
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Ego3S proposes a select-strengthen-synchronize framework to make large vision-language models reason efficiently over egocentric video.
Problem
Egocentric reasoning in large vision-language models is harder than third-person understanding because decisive cues are localized, atypical and scenes are highly dynamic.
Approach
Frames egocentric reasoning as needing focus on localized decisive cues, and proposes a select, strengthen, and synchronize pipeline to achieve efficient egocentric reasoning; the abstract does not specify details.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add coarse counterfactual filter (CCF) to full-data training (ablation) | Qwen2.5-VL-7B GRPO · QAEgo4D | pass@1: 63.80 -> 64.80 | Improved |
| add fine counterfactual filter (FCF) to full-data training (ablation) | Qwen2.5-VL-7B GRPO · QAEgo4D | pass@1: 63.80 -> 64.60 | Improved |
| combine CCF and FCF (full CSM) (ablation) | Qwen2.5-VL-7B GRPO · QAEgo4D | pass@1: 63.80 -> 65.80 | Improved |
| add interaction reward rint on top of CSM (ablation) | CSM · QAEgo4D | pass@1: 65.80 -> 66.20 | Improved |
| add penalty term rp on top of CSM+rint (ablation) | CSM+rint · EgoTextVQA | pass@1: 21.60 -> 22.00 | Improved |
| add Evolving Curriculum Recuration (ECR) (ablation) | CSM+IRL · EgoTextVQA | pass@1: 22.00 -> 22.40 | Improved |
| replace full-data training with CSM selection (26.5% of data) | GRPO · Egoplan2 | pass@1: 30.43 -> 33.76 | Improved |
| CSM selection instead of full data: training time | GRPO · Training cost | time (h): 60.5 -> 29.7 | Improved |
| use CSM instead of LIMR data selection | Qwen2.5-VL-7B · Egoplan2 | pass@1: 31.46 -> 33.76 | Improved |
| use IRL instead of GRPO | Qwen2.5-VL-7B · Egoplan2 | pass@1: 30.43 -> 35.42 | Improved |
| use IRL instead of DAPO | Qwen2.5-VL-7B · Egoplan2 | pass@1: 30.87 -> 35.42 | Improved |
| use RLOO instead of GRPO | Qwen2.5-VL-7B · EgoSchema | pass@1: 47.60 -> 46.90 | Hurt |
Closest papers by meaning
- EgoThinker: Unveiling Egocentric Reasoning with Spatio-Temporal CoT (NeurIPS 2025 · similarity 0.90)
- EgoProx: Evaluating MLLMs on Egocentric 3D Proximity Reasoning Across a Cognitive Hierarchy (CVPR 2026 · similarity 0.90)
- EARL: Towards a Unified Analysis-Guided Reinforcement Learning Framework for Egocentric Interaction Reasoning and Pixel Grounding (ICML 2026 · similarity 0.90)
- ANNEXE: Unified Analyzing, Answering, and Pixel Grounding for Egocentric Interaction (CVPR 2025 · similarity 0.89)
- EgoExoBench: A Benchmark for First- and Third-person View Video Understanding in MLLMs (NeurIPS 2025 · similarity 0.87)
- EgoChoir: Capturing 3D Human-Object Interaction Regions from Egocentric Views (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
