Research map / Vision-language and multimodal / Multimodal LLMs

RE-VLM: Event-Augmented Vision-Language Model for Scene Understanding

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RE-VLM, a dual-stream RGB+event vision-language model built on Qwen2.5-VL-3B with a graph-driven RGB-Event-Text data pipeline, releasing PEOD-Chat and RGBE-Chat benchmarks.

Problem

Standard VLMs fail under adverse imaging conditions (low light, HDR, fast motion), while event-only VLMs lack color and texture; there is also scarce RGB-Event-Text supervision for training and evaluation.

Approach

A dual-stream VLM fusing RGB and event encoders with a training-only spatio-temporal alignment module (STAM) and a relation loss, trained in three progressive stages, plus a graph-driven degradation-adaptive pipeline that fuses event and RGB scene graphs to synthesize captions and VQA.

Key results

• RE-VLM (3B backbone, reported as 4B) reaches Caption CI/DO/CU of 3.68/3.12/3.95 and VQA Acc 0.63 on PEOD-Chat, above the best RGB-only Qwen2.5-VL* (3.23/2.74/3.51, Acc 0.55).
• On RGBE-Chat, RE-VLM scores Ave 4.20 and Acc 0.75 versus 3.86 and 0.65 for fine-tuned Qwen2.5-VL*.
• In a manual audit on 855 PEOD samples, the graph pipeline's correction rate was 18.1% versus 54.2% for an RGB-only baseline.

Surprising findings

• The RGB-only Qwen2.5-VL baseline underperforms its own fine-tuned variant considerably; the fine-tuned event-only model is also reported to outperform the RGB-only baseline on some VQA metrics on RGBE-Chat in the ablation (Event-only Ave 3.53 vs RGB-only 4.10 shows the opposite ordering, so this is

Limitations

• Evaluation relies on LLM judges (GPT-3.5-Turbo, Qwen3-Omni) with a small Likert scale, and human audit covers only a sample of 855 items.
• Comparisons use event images rendered for all competing models, which may not suit event-native baselines.
• Some ablation claims (e.g., STAM) show only marginal gains or maintenance of performance.

Experimental setup

Datasets: PEOD RGBE-ImageNet N-ImageNet ImageNet DSEC DDD17 RGBE-SEG MVSEC M3ED

Benchmarks: PEOD-Chat RGBE-Chat

Metrics: CI DO CU Ave Acc

Baselines: Qwen2.5-VL InternVL DeepSeek-VL2 LLaVA-1.5 EventGPT

Builds on: Qwen2.5-VL EventGPT EventCLIP EventBind

Closest papers by meaning

• EventGPT: Event Stream Understanding with Multimodal Large Language Models (CVPR 2025 · similarity 0.91)
• Learning to See through Illumination Extremes with Event Streaming in Multimodal Large Language Models (CVPR 2026 · similarity 0.91)
• Talk2Event: Grounded Understanding of Dynamic Scenes from Event Cameras (NeurIPS 2025 · similarity 0.89)
• Synthesize Diagnose and Optimize: Towards Fine-Grained Vision-Language Understanding (CVPR 2024 · similarity 0.88)
• From Pixels to Words -- Towards Native Vision-Language Primitives at Scale (ICLR 2026 · similarity 0.88)
• ReVisionLLM: Recursive Vision-Language Model for Temporal Grounding in Hour-Long Videos (CVPR 2025 · similarity 0.88)