Research map / Detection and segmentation / Open-set and few-shot detection
Object-Centric Representation Learning for Enhanced 3D Semantic Scene Graph Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a contrastively pretrained, discriminative object feature encoder (cross-modal image/text supervised contrastive learning) plus a relationship encoder with LSE, bidirectional edge gating and global spatial attention, improving 3D semantic scene graph prediction on 3DSSG.
Problem
3D semantic scene graph prediction relies on GNNs over weakly discriminative object features, so object misclassification propagates into predicate errors; relationship features also underuse object semantics and geometry.
Approach
Pretrain an object encoder separately with a supervised contrastive loss aligning 3D object point clouds to their 2D CLIP-selected RGB views and CLIP text prompts (omitting the positive term from the denominator), then fuse object-pair semantics with geometric descriptors in a relationship encoder using LSE, GSE (distance-biased attention) and BEG (directional edge gating).
Key results
- On the 3DSSG validation set, the full model reaches 59.53 object R@1 and 91.27 predicate R@1, versus 55.93 and 89.81 for VL-SAT.
- Plugging the pretrained object encoder into VL-SAT improves its object R@1 from 55.68 to 59.30 and PredCls R@50 (w/ GC) from 80.0 to 81.7.
- Without graph constraints, PredCls R@100 reaches 94.6 versus 92.5 for VL-SAT, and SGCls R@100 reaches 52.5 versus 47.0.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add object feature learning (OFL) to SGPN (ablation) | SGPN · 3DSSG | Object R@1: 47.37 -> 54.49; +7.12% | Helped |
| Add object feature learning (OFL) to SGFN (ablation) | SGFN · 3DSSG | Object R@1: 56.18 -> 58.75; +2.57% | Helped |
| Add object feature learning (OFL) to VL-SAT (ablation) | VL-SAT · 3DSSG | Object R@1: 55.68 -> 59.30; +3.62% | Helped |
| Add Global Spatial Enhancement (GSE) (ablation) | Proposed model · 3DSSG (without graph constraints) | Triplet R@50: 90.19 -> 91.51 | Helped |
| Add Bidirectional Edge Gating (BEG) after GSE (ablation) | Proposed model · 3DSSG (without graph constraints) | Triplet mR@50: 62.59 -> 64.83 | Helped |
| Add Local Spatial Enhancement (LSE) after GSE and BEG (ablation) | Proposed model · 3DSSG (without graph constraints) | Predicate mR@1: 53.81 -> 56.32 | Helped |
Limitations
- Restricted to the closed-vocabulary 3DSSG setting; assumptions not directly compatible with open-vocabulary settings.
- Relies on GNN-based design rather than VLM/LLM-based approaches.
Experimental setup
Datasets: 3DSSG 3RScan
Benchmarks: 3DSSG
Metrics: Recall@K (object, predicate, triplet) SGCls R@K PredCls R@K mean recall
Baselines: SGPN SGFN VL-SAT Zhang et al.
Builds on: VL-SAT CrossPoint CLIP^2 PointNet SGFN
Closest papers by meaning
- Edge-Centric Relational Reasoning for 3D Scene Graph Prediction (AAAI 2026 · similarity 0.91)
- Open3DSG: Open-Vocabulary 3D Scene Graphs from Point Clouds with Queryable Objects and Open-Set Relationships (CVPR 2024 · similarity 0.91)
- Knowledge-inspired 3D Scene Graph Prediction in Point Cloud (NeurIPS 2021 · similarity 0.91)
- Learning 3D Semantic Scene Graphs From 3D Indoor Reconstructions (CVPR 2020 · similarity 0.90)
- DSGG: Dense Relation Transformer for an End-to-end Scene Graph Generation (CVPR 2024 · similarity 0.89)
- GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphs (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.