Research map / Video / Video-language understanding
Benchmarking Trustworthiness in Multimodal LLMs for Video Understanding
Youze Wang, Zijun Chen, Ruoyu Chen, Shishen Gu, Wenbo Hu, Jiayang Liu, Yinpeng Dong, Hang Su, Jun Zhu, Meng Wang, Richang Hong
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| closed-source vs open-source models | videoLLMs · Trust-videoLLMs overall | overall rank: Claude4-sonnet 1, Claude3.7 2 vs most open-source in lower half | Improved |
| scale parameters from 7B to 72B | videoLLMs · Trust-videoLLMs | overall rank: Qwen2.5-VL-7B rank 9 vs LLaVA-Video-72B rank 15 | No clear effect |
| scale model size | LongVA-7B vs LLaVA-Video-72B · event understanding | rank | No clear effect |
| stronger multimodal interaction robustness | videoLLMs · overall trustworthiness | rank correlation: r = 0.721 | Improved |
| adversarial robustness and OOD robustness | videoLLMs · overall trustworthiness | rank correlation: r = 0.019 and r = 0.027 | No clear effect |
| video-based jailbreak vs static image jailbreak | videoLLMs · jailbreak attacks | attack effectiveness | Hurt |
| open-source specialized optimization vs closed-source | videoLLMs · truthfulness | reasoning vs hallucination | Mixed |
Closest papers by meaning
- MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Models (NeurIPS 2024 · similarity 0.90)
- VMDT: Decoding the Trustworthiness of Video Foundation Models (NeurIPS 2025 · similarity 0.89)
- MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding (NeurIPS 2024 · similarity 0.88)
- Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis (CVPR 2025 · similarity 0.88)
- SF2T: Self-supervised Fragment Finetuning of Video-LLMs for Fine-Grained Understanding (CVPR 2025 · similarity 0.87)
- MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Models (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
