Research map / Vision-language and multimodal / Cross-modal learning
FusionNeXt: Sequence-First 3D Multi-Modal Fusion in the Era of LLMs
Yu Hong, Xiaosong Jia, Songbur Wong, Yanhao Liu, Wenlong Liao, Tao He, Junchi Yan
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FusionNeXt, a sequence-first approach to 3D multi-modal fusion that represents heterogeneous inputs as a single token sequence, motivated by the unified LLM/VLM architecture.
Problem
Multi-modal fusion for 3D perception has not adopted the sequence-first interface that modern LLMs/VLMs use to process heterogeneous inputs as one token sequence with generic attention and feed-forward operators.
Approach
Applies the sequence-first paradigm (heterogeneous inputs as a single token sequence processed by a generic deep stack of attention and feed-forward layers) to 3D multi-modal fusion. Details are not available in the abstract.
Closest papers by meaning
- VeXKD: The Versatile Integration of Cross-Modal Fusion and Knowledge Distillation for 3D Perception (NeurIPS 2024 · similarity 0.88)
- Inst3D-LMM: Instance-Aware 3D Scene Understanding with Multi-modal Instruction Tuning (CVPR 2025 · similarity 0.87)
- Revisiting Multimodal Fusion for 3D Anomaly Detection from an Architectural Perspective (AAAI 2025 · similarity 0.87)
- TAMM: TriAdapter Multi-Modal Learning for 3D Shape Understanding (CVPR 2024 · similarity 0.87)
- Coupled Mamba: Enhanced Multimodal Fusion with Coupled State Space Model (NeurIPS 2024 · similarity 0.86)
- MVD-Fusion: Single-view 3D via Depth-consistent Multi-view Generation (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
