Research map / 3D vision / 3D generation
REALM: An MLLM-Agent Framework for Open World 3D Reasoning Segmentation and Editing on Gaussian Splatting
Changyue Shi, Minghao Chen, Yiping Mao, Chuxiao Yang, Xinyuan Hu, Jiajun Ding, Zhou Yu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Qwen2.5-VL reasoning to GS-Group (ablation) | GS-Group · LERF Figurines | mIoU: 0.32 -> 0.78 | Improved |
| add global reasoning (multi-view parallel MLLM voting) (ablation) | REALM · LERF Figurines | mIoU: 0.78 -> 0.89 | Improved |
| add local refinement of the 3D mask (ablation) | REALM · LERF Figurines | mIoU: 0.89 -> 0.95 | Improved |
| add K-means clustering of camera poses for global view selection (ablation) | REALM · LERF Figurines | mIoU: 0.38 -> 0.95 | Improved |
| use Top-K-ID selection instead of random choice among K-means views (ablation) | REALM · LERF Figurines | mIoU: 0.76 -> 0.95 | Improved |
| use K-means + Top-K-ID instead of totally random global views (ablation) | REALM · LERF Figurines | mIoU: 0.59 -> 0.95 | Improved |
| vary number of K-means clusters (2, 24, 128) (ablation) | REALM · LERF Figurines | mIoU: 0.76 (2) / 0.95 (24) / 0.56 (128) | Mixed |
| increase number of global cameras from 4 to 8 (ablation) | REALM · LERF Figurines | mIoU: 0.81 -> 0.95 | Improved |
| increase number of global cameras from 8 to 16 (ablation) | REALM · LERF Figurines | mIoU: 0.95 -> 0.95 | No clear effect |
| increase local refinement steps from 50 to 500 or 1000 (ablation) | REALM · LERF Figurines | mIoU: 0.95 -> 0.79 (500), 0.74 (1000) | Hurt |
| reduce local refinement steps from 50 to 10 (ablation) | REALM · LERF Figurines | mIoU: 0.95 -> 0.94 | No clear effect |
| replace MLLM aggregation with random views fed directly to the MLLM (ablation) | REALM · LERF Figurines | mIoU | Hurt |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- GAGS — GAGS: Granularity-Aware Feature Distillation for Language Gaussian Splatting (AAAI 2026) · better on 5 of 5 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- MLLM-For3D: Adapting Multimodal Large Language Model for 3D Reasoning Segmentation (NeurIPS 2025 · similarity 0.89)
- ReasonGrounder: LVLM-Guided Hierarchical Feature Splatting for Open-Vocabulary 3D Visual Grounding and Reasoning (CVPR 2025 · similarity 0.89)
- Abstract 3D Perception for Spatial Intelligence in Vision-Language Models (CVPR 2026 · similarity 0.89)
- S$^2$-MLLM: Boosting Spatial Reasoning Capability of MLLMs for 3D Visual Grounding with Structural Guidance (CVPR 2026 · similarity 0.88)
- Imagine3D-LLM: Teaching MLLMs to Imagine 3D Scenes Before Answering (NeurIPS 2026 · similarity 0.88)
- Pursuing Minimal Sufficiency in Spatial Reasoning (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
