Research map / Vision-language and multimodal / Cross-modal learning
REALM: An RGB and Event Aligned Latent Manifold for Cross-Modal Perception
Vincenzo Polizzi, David Lindell, Jonathan Kelly
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| align event features to frozen DUNE with LoRA and use frozen MASt3R head | REALM · ECD event matching | AUC@20: 46.7 -> 63.3 | Improved |
| use REALM vs MINIMA on VECtor | REALM · VECtor event-event | AUC@5: 39.0 -> 47.7 | Improved |
| event student vs RGB teacher on low-light depth | REALM · MVSEC outdoor night 1 (20 m) | depth error (m): 3.10 -> 2.51 | Improved |
| event student vs RGB teacher on daytime depth | REALM · MVSEC outdoor day 1 (20 m) | depth error (m): 1.76 -> 2.42 | Hurt |
| REALM with linear head vs specialized transformer decoders on depth | REALM · MVSEC extended | AbsRel: EReFormer 0.275 vs REALM 0.349; latency 35.17 vs 6.71 ms | Mixed |
| REALM vs specialized event segmentation | REALM · DSEC segmentation | mIoU: 57.55 (ESEG) vs 55.37; ESS 51.57 | Mixed |
| REALM event student vs RGB teacher on segmentation | REALM · DSEC segmentation | mIoU: 67.64 -> 55.37 | Hurt |
| add progressive spatial masking of distillation loss (ablation) | REALM · segmentation and depth | mIoU / RMSE: mIoU 53.97 -> 55.47; RMSE 8.42 -> 8.06 | Improved |
| apply LoRA to attention and feed-forward layers instead of attention only (ablation) | REALM · EPM segmentation | mIoU: 10.06 -> 11.69 | Improved |
| same, depth (ablation) | REALM · EPM depth | RMSE: 10.26 -> 10.55 | Hurt |
| LoRA rank 32 vs 16 or 8 (ablation) | REALM · EPM | mIoU: 10.82 (r=8) / 11.18 (r=16) -> 11.69 | Improved |
| use voxel grid instead of Tencode or ERGO input (ablation) | REALM · DSEC segmentation | IoU: 50.64 / 50.62 -> 53.75 | Improved |
Closest papers by meaning
- RE-VLM: Event-Augmented Vision-Language Model for Scene Understanding (CVPR 2026 · similarity 0.87)
- Learning to See through Illumination Extremes with Event Streaming in Multimodal Large Language Models (CVPR 2026 · similarity 0.87)
- EventDance: Unsupervised Source-free Cross-modal Adaptation for Event-based Object Recognition (CVPR 2024 · similarity 0.86)
- Re-coding for Uncertainties: Edge-awareness Semantic Concordance for Resilient Event-RGB Segmentation (NeurIPS 2025 · similarity 0.86)
- The Shape of Events: Edge-Based Inductive Biases via Cross-Domain Distillation (NeurIPS 2026 · similarity 0.86)
- EventGPT: Event Stream Understanding with Multimodal Large Language Models (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
