Research map / 3D vision / 3D generation
Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spaces
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces functional 3D scene graphs that link objects to interactive elements via functional relationships, and an open-vocabulary, training-free foundation-model pipeline (OpenFunGraph) to build them from posed RGB-D, plus a new FunGraph3D dataset.
Problem
Existing 3D scene graphs model only objects and spatial edges, missing small interactive elements (handles, switches) and their functional relationships; there is little training data for functional relations.
Approach
Training-free open-vocabulary pipeline that detects objects and interactive elements with RAM++ and GroundingDINO, describes nodes with LLaVA and GPT-4, and infers functional edges via sequential local-then-remote reasoning with confidence scores.
Key results
- OpenFunGraph reaches 73.0 overall node R@3 and 60.4 triplet R@5 on SceneFun3D, versus 60.1 and 34.3 for ConceptGraph* + IED.
- On FunGraph3D, OpenFunGraph reaches 29.8 triplet R@5 versus 10.3 for ConceptGraph* + IED and 7.3 for Open3DSG*†.
- Ablations show removing sequential edge reasoning drops SceneFun3D triplet R@5 from 60.4 to 34.8.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add assistive object tags to interactive-element detection prompts (ablation) | OpenFunGraph · SceneFun3D | overall node recall R@3: 59.3 -> 73.0 | Helped |
| Add assistive object tags to interactive-element detection prompts (ablation) | OpenFunGraph · FunGraph3D | overall node recall R@3: 49.9 -> 55.5 | Helped |
| Add assistive object tags to interactive-element detection prompts (ablation) | OpenFunGraph · SceneFun3D | overall triplet recall R@5: 48.3 -> 60.4 | Helped |
| Add assistive object tags to interactive-element detection prompts (ablation) | OpenFunGraph · FunGraph3D | overall triplet recall R@5: 23.1 -> 29.8 | Helped |
| Use sequential rather than direct edge reasoning across all nodes (ablation) | OpenFunGraph · SceneFun3D | overall triplet recall R@5: 34.8 -> 60.4 | Helped |
| Use sequential rather than direct edge reasoning across all nodes (ablation) | OpenFunGraph · FunGraph3D | overall triplet recall R@5: 20.2 -> 29.8 | Helped |
| Use confidence-aware remote edge selection instead of random connection selection (ablation) | OpenFunGraph · SceneFun3D | overall triplet recall R@5: 56.0 -> 60.4 | Helped |
| Use confidence-aware remote edge selection instead of random connection selection (ablation) | OpenFunGraph · FunGraph3D | overall triplet recall R@5: 26.8 -> 29.8 | Helped |
Surprising findings
- Plain ConceptGraph with only functional prompts recovers almost no interactive elements (overall node R@3 on objects-only ConceptGraph* is 6.6 on SceneFun3D).
- Open3DSG-based baselines struggle with functional relations because their edge features rely on spatial adjacency.
Limitations
- Evaluation restricted to visually unambiguous functional relationships because annotators could not physically access scenes.
- FunGraph3D is small (14 scenes), and triplet recall remains low on FunGraph3D (29.8 R@5).
Experimental setup
Datasets: SceneFun3D FunGraph3D
Benchmarks: SceneFun3D FunGraph3D
Metrics: Recall@K for nodes Recall@K for triplets node association recall edge prediction recall
Baselines: Open3DSG ConceptGraph IFR-Explore
Builds on: ConceptGraph Open3DSG chain-of-thought prompting GroundingDINO RAM++
Closest papers by meaning
- SceneFun3D: Fine-Grained Functionality and Affordance Understanding in 3D Scenes (CVPR 2024 · similarity 0.91)
- Open3DSG: Open-Vocabulary 3D Scene Graphs from Point Clouds with Queryable Objects and Open-Set Relationships (CVPR 2024 · similarity 0.90)
- InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Prior (ICLR 2024 · similarity 0.89)
- TB-HSU: Hierarchical 3D Scene Understanding with Contextual Affordances (AAAI 2025 · similarity 0.88)
- Functionality Understanding and Segmentation in 3D Scenes (CVPR 2025 · similarity 0.88)
- SPARTUN3D: Situated Spatial Understanding of 3D World in Large Language Model (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.