Research map / Vision-language and multimodal / Multimodal LLMs
VidUEU-Agent: A Data-Curation Agent for Multimodal Understanding, Editing, and Unified Tasks
Tengjv Ru, Weitong Lian, Zecong Tang, Lingyi Meng, Haoran Li, Zhejun Cui, Yichen Zhu, Hangshuo Cao, Qi Kang, Yechi Liu, Kaixuan Wang, Yu-Jie Yuan and 3 more
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Presents VidUEU-Agent, an automated agent pipeline that synthesizes multimodal instruction data from raw videos for understanding, editing, and unified tasks.
Problem
Scarcity and high annotation cost of high-quality multimodal instruction data, while raw videos are abundant but hard to convert into precise instruction data.
Approach
An agent-based automated pipeline that curates instruction data directly from raw videos to cover multimodal understanding, editing, and unified tasks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune on VidUEU-100K understanding data | InternVL3-8B · MMDU | score: 53.1 -> 61.1 | Improved |
| fine-tune on VidUEU-100K understanding data | InternVL3-8B · MME | score: 2308.0 -> 2369.3 | Improved |
| fine-tune on VidUEU-100K understanding data | InternVL3-8B · LLaVA-Bench | score: 69.0 -> 70.8 | No clear effect |
| fine-tune on VidUEU-100K editing data | Bagel · GEdit-Bench-En G_PQ | score: 6.831 -> 7.328 | Improved |
| fine-tune on VidUEU-100K unified data | Bagel · WEAVEbench Overall | score: 0.446 -> 0.501 | Improved |
| add autonomous data-distribution iteration | VidUEU-Agent · WEAVEbench | score: 0.448 -> 0.526 | Improved |
| update data distribution from iteration 1 to iteration 2 | VidUEU-Agent · MMDU | score: 29.7 -> 29.2 | Hurt |
| add Gate B frame quality filtering (ablation) | VidUEU-Agent · GEdit-Bench-En G_O | score: 5.175 -> 6.951 | Improved |
| add the agent pipeline with tool evidence versus fixed-interval VLM QA (ablation) | VidUEU-Agent · MMDU | score: 22.0 -> 33.2 | Improved |
| add Gate A shot filtering (ablation) | VidUEU-Agent · WEAVEbench Overall | score: 0.497 -> 0.501 | No clear effect |
| add score-and-similarity greedy keyframe sampling (ablation) | VidUEU-Agent · GEdit-Bench-En G_O | score: 5.848 -> 6.951 | Improved |
| add consistency hard rule requiring signal ID citation (ablation) | VidUEU-Agent · MMDU | score: 29.4 -> 33.2 | Improved |
Experimental setup
Datasets: raw videos
Closest papers by meaning
- VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Tasks (ICLR 2025 · similarity 0.88)
- DeepEyesV2: Toward Agentic Multimodal Model (ICLR 2026 · similarity 0.87)
- OmniGen2: Towards Instruction-Aligned Multimodal Generation (CVPR 2026 · similarity 0.87)
- VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agents (ICLR 2025 · similarity 0.87)
- AgentVista: Evaluating Multimodal Agent in Ultra-Challenging Realistic Visual Scenarios (NeurIPS 2026 · similarity 0.87)
- AgenticVBench: Can AI Agents Complete Real-World Video Production Tasks? (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
