Research map / Multi-agent systems and games / Human-AI and goal-conditioned
LEMMA: A Multi-view Dataset for LEarning Multi-agent Multi-task Activities
Baoxiong Jia, Yixin Chen, Siyuan Huang, Yixin Zhu, Song-Chun Zhu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| predict verbs and nouns separately (branching) instead of compositional action directly | SlowFast · LEMMA FPV recognition | compositional action Avg.F1: 27.3 -> 35.58 | Improved |
| predict verbs and nouns separately (branching) instead of compositional action directly | I3D · LEMMA FPV recognition | compositional action Avg.F1: 17.30 -> 30.53 | Improved |
| use sequential inference instead of branching (ablation) | SlowFast · LEMMA FPV recognition | compositional action Avg.F1: 35.58 -> 31.57 | Hurt |
| predict verbs and nouns separately instead of compositional action directly | SlowFast · LEMMA TPV recognition | compositional action Avg.F1: 16.31 -> 10.01 | Hurt |
| predict verbs and nouns separately (branching) instead of directly | I3D · LEMMA TPV recognition | compositional action Avg.F1: 10.64 -> 7.6 | No clear effect |
| use FPV view instead of TPV view | SlowFast · LEMMA recognition | compositional action Avg.F1: 16.31 -> 27.3 | Improved |
| add long-term feature bank temporal context to segment features (ablation) | SlowFast · LEMMA action anticipation 1x1 | compositional action Avg.F1: 22.82 -> 25.54 | Improved |
| add primary TPV features to LFB (ablation) | LFB · LEMMA task anticipation 1x1 | task Avg.F1: 68.42 -> 69.54 | Improved |
| add primary TPV features to LFB (ablation) | LFB · LEMMA action anticipation 1x1 | compositional action Avg.F1: 25.54 -> 29.01 | Improved |
| add other agent's TPV-fused features to LFB (ablation) | LFB · LEMMA task anticipation 2x1 | task Avg.F1: 44.56 -> 51.23 | Improved |
| add other agent's TPV-fused features to LFB (ablation) | LFB · LEMMA task anticipation 2x2 | task Avg.F1: 55.19 -> 56.67 | Improved |
| add other agent's FPV features to SF (ablation) | SF · LEMMA task anticipation 2x1 | task Avg.F1: 46.49 -> 48.08 | Improved |
Closest papers by meaning
- Human-Centric Open-Future Task Discovery: Formulation, Benchmark, and Scalable Tree-Based Search (AAAI 2026 · similarity 0.87)
- ProAct: A Benchmark and Multimodal Framework for Structure-Aware Proactive Response (ICML 2026 · similarity 0.87)
- FineVLA: Fine-Grained Instruction Alignment for Steerable Vision-Language-Action Policies (NeurIPS 2026 · similarity 0.86)
- From Isolated Islands to Pangea: Unifying Semantic Space for Human Action Understanding (CVPR 2024 · similarity 0.86)
- PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Tasks (ICLR 2025 · similarity 0.86)
- LLaVAction: evaluating and training multi-modal large language models for action understanding (ICLR 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
