Research map / Video / Video-language understanding
UTOPI: Efficient Egocentric Long-Video Understanding in AR via User-Guided Token Pre-Compression
ziqi wang, Su Chen, Qiance Tang, Jieyu Lin, Ziyun Li, Barbara De Salvo, Sai Qian Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UTOPI, a plug-and-play user-guided token pre-compression module that reduces redundant visual tokens in egocentric long videos for resource-limited AR devices.
Problem
Egocentric long-video understanding on AR devices is constrained by heavy memory and compute demands caused by substantial temporal redundancy in first-person videos.
Approach
A plug-and-play pre-compression module that uses user guidance (e.g., the wearer's attention/interaction cues) to pre-compress visual tokens before they reach the video model, tailored to egocentric streams.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply UTOPI with 15% token budget | Qwen2.5VL-7B · EgoEye, EgoSchema, EgoGazeVQA | accuracy (%): EE 58.4 -> 60.6; ES 60.1 -> 64.8; EGV 47.8 -> 49.0 | Improved |
| use random token drop at 15% budget | Qwen2.5VL-7B · EgoEye, EgoSchema, EgoGazeVQA | accuracy (%): EE 58.4 -> 56.8; ES 60.1 -> 59.3; EGV 47.8 -> 46.1 | Hurt |
| use frame subsampling at 15% budget | Qwen2.5VL-7B · EgoEye | accuracy (%): 58.4 -> 48.8 | Hurt |
| use UTOPI instead of DTD/PruneVid at 15% budget | token compression · EgoEye | accuracy (%): 57.6/51.3 -> 60.6 | Improved |
| add UTOPI to Qwen2.5VL-7B at extreme budget 3% | Qwen2.5VL-7B · EgoEye | accuracy (%): 58.4 -> 50.5 | Hurt |
| add UTOPI at 5% budget | Qwen2.5VL-7B · EgoSchema | accuracy (%): 60.1 -> 61.4 | Improved |
| add UTOPI to PLLaVA at 15% budget | PLLaVA · EgoEye | accuracy (%): 34.8 -> 33.5 | Hurt |
| enable AGTS alone (ablation) | UTOPI · EgoSchema | accuracy (%): 60.1 -> 64.1 | Improved |
| enable IVOC alone (ablation) | UTOPI · EgoSchema | accuracy (%): 60.1 -> 64.4 | Improved |
| enable both AGTS and IVOC (ablation) | UTOPI · EgoSchema | accuracy (%) and token budget: 64.4 -> 64.8 at 15% budget | Improved |
| inject gaze noise up to 300 px (ablation) | UTOPI · EgoEye | accuracy (%): 60.6 -> 57.7 | Hurt |
| replace ground-truth gaze with SUM-predicted gaze (ablation) | UTOPI · EgoEye | accuracy (%): 60.6 -> 59.4 | No clear effect |
Closest papers by meaning
- Grounded-Exo2Ego: Structured Semantic Grounding for Robust Exocentric-to-Egocentric Video Generation (NeurIPS 2026 · similarity 0.85)
- Action Scene Graphs for Long-Form Understanding of Egocentric Videos (CVPR 2024 · similarity 0.85)
- Object-Shot Enhanced Grounding Network for Egocentric Video (CVPR 2025 · similarity 0.85)
- Viewpoint Rosetta Stone: Unlocking Unpaired Ego-Exo Videos for View-invariant Representation Learning (CVPR 2025 · similarity 0.84)
- Retrieval-Augmented Egocentric Video Captioning (CVPR 2024 · similarity 0.84)
- EgoVid-5M: A Large-Scale Video-Action Dataset for Egocentric Videos Generation (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
