Research map / 3D vision / 3D generation
SceneFun3D: Fine-Grained Functionality and Affordance Understanding in 3D Scenes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SceneFun3D, a laser-scan-based dataset of 710 indoor scenes with fine-grained functional interactive element masks, affordance labels, motion parameters and task descriptions, plus three benchmark tasks adapted from existing 3D segmentation methods.
Problem
Existing 3D scene understanding focuses on semantic and instance segmentation of objects, lacking fine-grained annotations of functional interactive elements (handles, knobs, buttons) and their affordances and motions in real-world 3D scenes, which are often missed by commodity RGB-D reconstructions
Approach
Annotate functional interactive elements directly on high-resolution Faro laser scans (voxel 5mm) with Gibsonian affordance labels, motion parameters, and telic task descriptions linking affordances to purpose, and adapt existing segmentation and open-vocabulary methods to three new tasks.
Key results
- Mask3D-F achieves the best functionality segmentation result with AP 7.9, AP50 18.3, AP25 26.6, versus LERF at AP25 18.1 and SoftGroup-F at 17.2.
- OpenMask3D-F reaches AP25 17.5 on task-driven affordance grounding, outperforming LERF (11.3) and the original OpenMask3D (0.0).
- Motion estimation with Mask3D-FM using rgb+normal achieves AP25 26.5 and 8.1 when requiring type, axis, and origin all correct (+MAO).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Adapt Mask3D with fine-grained masks and curriculum learning | Functionality segmentation · Functionality segmentation test | AP25: 17.2 -> 26.6 | Helped |
| Use functional-element masks instead of object-level masks for language-guided segmentation | OpenMask3D · Task-driven affordance grounding test | AP25: 0.0 -> 17.5 | Helped |
| Use functional-element masks for grounding | OpenMask3D · Task-driven affordance grounding test | AP50: 0.0 -> 8.0 | Helped |
| Add coarse-to-fine mask expansion with r0=0.1 (ablation) | Mask3D-F · Functionality segmentation test | AP25: 0.0 -> 26.6 | Helped |
| Set initial mask expansion radius to 0.2 instead of 0.1 (ablation) | Mask3D-F · Functionality segmentation test | AP25: 26.6 -> 26.2 | Hurt |
| Set initial mask expansion radius to 0.05 instead of 0.1 (ablation) | Mask3D-F · Functionality segmentation test | AP25: 26.6 -> 18.6 | Hurt |
| Disable coarse-to-fine mask expansion (ablation) | Mask3D-F · Functionality segmentation test | AP25: 26.6 -> 0.0 | Hurt |
| Set crop-expansion ratio to 0.5 instead of 0.1 (ablation) | OpenMask3D-F · Task-driven affordance grounding test | AP50: 4.5 -> 8.3 | Helped |
| Set crop-expansion ratio to 0.5 instead of 0.1 (ablation) | OpenMask3D-F · Task-driven affordance grounding test | AP25: 11.2 -> 16.2 | Helped |
| Set crop-expansion ratio to 1.0 instead of 0.5 (ablation) | OpenMask3D-F · Task-driven affordance grounding test | AP50: 8.3 -> 8.0 | Hurt |
| Set crop-expansion ratio to 1.0 instead of 0.5 (ablation) | OpenMask3D-F · Task-driven affordance grounding test | AP25: 16.2 -> 17.5 | Helped |
| Set crop-expansion ratio to 2.0 instead of 1.0 (ablation) | OpenMask3D-F · Task-driven affordance grounding test | AP25: 17.5 -> 16.5 | Hurt |
Surprising findings
- Original OpenMask3D scores 0.0 AP on task-driven affordance grounding, showing object-level methods fail on functional elements.
- Disabling coarse-to-fine curriculum (r0=0) makes Mask3D-F fail entirely, scoring 0.0 AP50 and AP25.
Limitations
- Small absolute scores across all three tasks show the benchmarks remain challenging.
- Very small structures such as knobs on electrical devices are hard to segment.
- Reflective-material cases are excluded from evaluation under the 'exclude' label.
Experimental setup
Datasets: ARKitScenes SceneFun3D
Benchmarks: SceneFun3D
Metrics: AP AP50 AP25
Baselines: SoftGroup-F LERF Mask3D-F OpenMask3D
Builds on: Mask3D SoftGroup OpenMask3D LERF ARKitScenes
Closest papers by meaning
- Functionality Understanding and Segmentation in 3D Scenes (CVPR 2025 · similarity 0.91)
- TB-HSU: Hierarchical 3D Scene Understanding with Contextual Affordances (AAAI 2025 · similarity 0.91)
- Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spaces (CVPR 2025 · similarity 0.91)
- UniFunc3D: Unified Active Spatial-Temporal Grounding for 3D Affordance Segmentation (NeurIPS 2026 · similarity 0.90)
- SeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Language Model (CVPR 2025 · similarity 0.89)
- AffordBot: 3D Fine-grained Embodied Reasoning via Multimodal Large Language Models (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.