Research map / 3D vision / 3D generation
LLaVA-4D: Embedding SpatioTemporal Prompt into LMMs for 4D Scene Understanding
Hanyu Zhou, Gim Hee Lee
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add coordinate embedding (ablation) | LLaVA-4D · Chat4D | TAcc: 12.7 -> 47.5 | Improved |
| add feature disentanglement (ablation) | LLaVA-4D · Chat4D | CIDEr: 85.4 -> 89.0 | Improved |
| add feature fusion (ablation) | LLaVA-4D · Chat4D | TAcc: 51.2 -> 54.6 | Improved |
| add 3D position encoding alone (ablation) | LLaVA-4D · Chat4D | TAcc: 46.8 -> 47.0 | No clear effect |
| add 1D time encoding alone (ablation) | LLaVA-4D · Chat4D | TAcc: 46.8 -> 52.7 | Improved |
| use full 4D coordinate encoding instead of no encoding (ablation) | LLaVA-4D · Chat4D | CIDEr: 75.0 -> 93.5 | Improved |
| use attention fusion instead of concatenation (ablation) | LLaVA-4D · Chat4D | SAcc@0.5: 54.3 -> 58.9 | Improved |
| use attention fusion instead of weighting (ablation) | LLaVA-4D · Chat4D | TAcc: 51.4 -> 54.6 | Improved |
| add textual coordinates in instructions (ablation) | LLaVA-4D · Chat4D | TAcc: 25.8 -> 53.0 | Improved |
| add encoding of textual coordinates (ablation) | LLaVA-4D · Chat4D | CIDEr: 90.1 -> 93.5 | Improved |
| use LLaVA-4D instead of Video-3D LLM on ScanQA | LLaVA-4D · ScanQA | CIDEr: 102.1 vs 97.8 | Hurt |
| use LLaVA-4D instead of Video-3D LLM on Scan2Cap | LLaVA-4D · Scan2Cap | CIDEr@0.5: 83.8 -> 85.3 | Improved |
Closest papers by meaning
- Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understanding (CVPR 2025 · similarity 0.89)
- GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models (ICLR 2026 · similarity 0.88)
- ConsisVLA-4D: Advancing Spatiotemporal Consistency in Efficient 3D-Perception and 4D-Reasoning for Robotic Manipulation (CVPR 2026 · similarity 0.88)
- Motion4D: Learning 3D-Consistent Motion and Semantics for 4D Scene Understanding (NeurIPS 2025 · similarity 0.88)
- 4D-VLA: Spatiotemporal Vision-Language-Action Pretraining with Cross-Scene Calibration (NeurIPS 2025 · similarity 0.88)
- 3D-LLaVA: Towards Generalist 3D LMMs with Omni Superpoint Transformer (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
