Research map / 3D vision / 3D generation

ElasticFit: Fit-Aware 3D Object Insertion via VLM Reasoning and Generative Adaptation

NeurIPS 2026 · 3D vision › 3D generation · approach: Open-vocabulary 3D scene understanding

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ElasticFit, a VLM-guided pipeline that grounds object insertion in a scene via structured fitting cues, generates and lifts a scene-conditioned object, and refines it with rigid/uniform/elastic fitting plus physics settling.

Problem

Inserting objects into pre-existing 3D scenes requires fitting the object to local geometry, support, and free space, which fixed-asset placement methods and layout-level VLM methods handle poorly.

Approach

A scene-grounded fitting representation G, inferred by a VLM from view-aligned renders with Set-of-Mark IDs, converts language into support point, ghost box, orientation, and adaptation mode; this conditions 2D-to-3D generation and mode-specific geometric fitting followed by physics settling.

Key results

• On the Common Placement Set, ElasticFit (placement-only) reaches 69.7% SRS and 91.7% SupR versus the best baseline LayoutGPT at 50.8% SRS and 48.3% SupR.
• Full ElasticFit on the 60-task Fit-Critical Set achieves 60.8% SRS, 71.7% CFR, 83.3% SupR, and CPS 4.57.
• Removing geometry-physics fitting on gap-fitting tasks drops CFR from 76.7% to 36.7% and raises penetrating vertex ratio from 1.08% to 6.09%.

Surprising findings

• Holodeck and LayoutVLM have the highest collision-free rates (95.0% and 93.0%) but far lower SRS and SupR, reflecting conservative placement far from scene geometry.
• On gap-fitting tasks, 24 of 25 SRS-unsatisfied cases still receive CPS of at least 4, suggesting strict predicate checks penalize plausible results.

Limitations

• Elastic mode uses only anisotropic scaling, not true non-rigid deformation.
• Objects can be left partially hanging over surface edges; containment is not enforced.
• Evaluation task suite was developed alongside the method; repeated-run variability not quantified.

Experimental setup

Datasets: ReplicaCAD Replica

Benchmarks: Common Placement Set Fit-Critical Set

Metrics: SRS CFR SupR CPS penetrating vertex ratio

Baselines: LayoutGPT Holodeck LayoutVLM

Builds on: Set-of-Mark TRELLIS SDXL FirePlace

Closest papers by meaning

• S$^2$-MLLM: Boosting Spatial Reasoning Capability of MLLMs for 3D Visual Grounding with Structural Guidance (CVPR 2026 · similarity 0.86)
• Let Language Constrain Geometry: Vision–Language Models as Semantic and Spatial Critics for 3D Generation (ICML 2026 · similarity 0.86)
• FirePlace: Geometric Refinements of LLM Common Sense Reasoning for 3D Object Placement (CVPR 2025 · similarity 0.86)
• MetaSpatial: Reinforcing 3D Spatial Reasoning in VLMs for the Metaverse (ICLR 2026 · similarity 0.86)
• LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Models (CVPR 2025 · similarity 0.86)
• InfiniBench: Infinite Benchmarking for Visual Spatial Reasoning with Customizable Scene Complexity (CVPR 2026 · similarity 0.85)