Research map / Vision-language and multimodal / Human and embodied modeling
3M-Game: Multi-Modal Multi-Task Multi-Teacher Learning for Game Event Detection (Student Abstract)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes 3M-Game, a multi-teacher knowledge distillation framework that distills audio, chat, and transcript expert teachers into a RoBERTa student for LoL game event detection on Twitch/YouTube livestreams.
Problem
Detecting game events (kills, towers, dragons) in esports livestreams is hard for newcomers; prior work uses single-modality game data or concurrent multimodal training, leaving livestream audio, commentary, and chat under-integrated.
Approach
Fine-tune independent modality-specific teachers (AST for audio, XLM-RoBERTa for chat, RoBERTa for transcript) and distill their hidden states and soft labels into a single transcript-only student via multi-teacher hidden-state and distillation losses plus a task loss, so the student carries multimodal knowledge at inference.
Key results
- 3M-Game (All modalities) reaches precision 0.495 on all events vs 0.340 for the Game-MUG BERT-base baseline.
- On main events only, 3M-Game achieves 0.638 precision vs 0.185 for the baseline.
- Removing the chat teacher drops precision to 0.325 on all events, the largest ablation drop among the reported teachers.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use 3M-Game with audio, chat, and transcript modalities | 3M-Game · Game-MUG (All) | precision: 0.340 -> 0.495 | Helped |
| use 3M-Game with audio, chat, and transcript modalities | 3M-Game · Game-MUG (Mains Only) | precision: 0. 185 -> 0. 638 | Helped |
| add the audio modality (ablation) | 3M-Game · Game-MUG (All) | precision: 0.483 -> 0.495 | Helped |
| add the chat modality (ablation) | 3M-Game · Game-MUG (All) | precision: 0.325 -> 0.495 | Helped |
| add the transcript modality (ablation) | 3M-Game · Game-MUG (All) | precision: 0.447 -> 0.495 | Helped |
Limitations
- Evaluation limited to precision on a single dataset with a single baseline.
- Only a short abstract; no variance or significance reported.
- Ablation of teachers covers only one-at-a-time removal.
Experimental setup
Datasets: Game-MUG
Benchmarks: Game-MUG
Metrics: precision (macro)
Baselines: Game-MUG BERT-base baseline
Builds on: Game-MUG knowledge distillation
Closest papers by meaning
- Cross-modal Multi-task Learning for Multimedia Event Extraction (AAAI 2025 · similarity 0.82)
- Balancing Multimodal Training Through Game-Theoretic Regularization (NeurIPS 2025 · similarity 0.81)
- Towards Universal Soccer Video Understanding (CVPR 2025 · similarity 0.81)
- OmniMMI: A Comprehensive Multi-modal Interaction Benchmark in Streaming Video Contexts (CVPR 2025 · similarity 0.81)
- Can Multi-Modal LLMs Provide Live Step-by-Step Task Guidance? (NeurIPS 2025 · similarity 0.80)
- GlitchBench: Can Large Multimodal Models Detect Video Game Glitches? (CVPR 2024 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.