Research map / Video / Video segmentation and tracking
Ex2Eg-MAE: A Framework for Adaptation of Exocentric Video Masked Autoencoders for Egocentric Social Role Understanding
Minh Tran, Yelin Kim, Che-Chun Su, Min Sun, Cheng-Hao Kuo, Mohammad Soleymani
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add teacher distillation loss to reconstruction loss (ablation) | Ex2Eg-MAE · Ego4D Talk-to-me | mAP: 58.4 -> 59.5 | Improved |
| add perspective shift estimation (adversarial) loss (ablation) | Ex2Eg-MAE · Ego4D Talk-to-me | mAP: 59.5 -> 60.1 | Improved |
| weight mouth and eye pixels more in reconstruction loss (Wface) (ablation) | Ex2Eg-MAE · Ego4D Talk-to-me | mAP: 60.1 -> 60.5 | Improved |
| add distillation loss (ablation) | Ex2Eg-MAE · EasyCom Talk-to-me | mAP: 82.4 -> 84.0 | Improved |
| add perspective shift loss and face weighting (ablation) | Ex2Eg-MAE · EasyCom Talk-to-me | mAP: 84.0 -> 84.9 -> 85.4 | Improved |
| pretrain on synthetic EgoVox instead of adapting MARLIN on Ego4D | Ex2Eg-MAE · Ego4D Talk-to-me | mAP: 58.3 -> 60.5 | Improved |
| adapt VideoMAE/MARLIN with Ego4D self-supervised pretraining | VideoMAE · Ego4D Look-at-me | mAP: 68.2 -> 61.2 | Hurt |
| scale encoder from tiny to base to large (ablation) | Ex2Eg-MAE · Ego4D Look-at-me | mAP: 69.4 -> 73.9 -> 78.3 | Improved |
| fuse Whisper audio predictions with Ex2Eg-MAE video | Ex2Eg-MAE · Ego4D Talk-to-me | mAP: 60.5 -> 68.1 | Improved |
| use masking ratio near 75% rather than very high ratios | Ex2Eg-MAE · pretraining | downstream performance | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 2 of 2 on most shared results.
- MARLIN — MARLIN: Masked Autoencoder for Facial Video Representation LearnINg (CVPR 2023) · better on 3 of 3 shared results
- VideoMAE — VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training (NeurIPS 2022) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Bootstrap Your Own Views: Masked Ego-Exo Modeling for Fine-grained View-invariant Video Representations (CVPR 2025 · similarity 0.90)
- The Audio-Visual Conversational Graph: From an Egocentric-Exocentric Perspective (CVPR 2024 · similarity 0.88)
- EgoExoBench: A Benchmark for First- and Third-person View Video Understanding in MLLMs (NeurIPS 2025 · similarity 0.88)
- Retrieval-Augmented Egocentric Video Captioning (CVPR 2024 · similarity 0.87)
- Learning Spatial Features from Audio-Visual Correspondence in Egocentric Videos (CVPR 2024 · similarity 0.87)
- Action Scene Graphs for Long-Form Understanding of Egocentric Videos (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
