Research map / Graph learning / Graph self-supervised learning
SegGraph: Leveraging Graphs of SAM Segments for Few-Shot 3D Part Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SegGraph, a few-shot 3D part segmentation method that builds a graph over SAM segments from multi-view renderings (overlap and adjacency edges), propagates DINOv2 features with GATv2, and unpools to points with view-direction-weighted fusion.
Problem
Few-shot 3D part segmentation requires effectively transferring 2D foundation model knowledge to 3D while respecting geometric structure and SAM's grouping cues; existing label-aggregation and feature-aggregation methods under-segment parts, produce inconsistent labels, or lose small components thro
Approach
Construct a SAM-segment graph in 3D (nodes = view-consistent segments, edges = overlap across views or spatial adjacency), propagate segment features with GATv2 over the two edge types, and fuse segment features back to points with a learned viewing-direction quality weight.
Key results
- On PartNet-E (45 categories, 8-shot), SegGraph reaches 72.8% mIoU with DINOv2 and SAM, versus 66.7% for the best prior few-shot baseline PartDistill (per the paper's ~6.9% margin claim).
- On ShapeNetPart (16 categories), SegGraph achieves 62.6% mIoU versus 57.0% for PointCLIPv2.
- On six small-component categories, SegGraph reaches 49.8% mIoU on small parts versus 28.9% for PartSTAD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SegGraph with DINOv2 and SAM instead of PartDistill | PartNet-E few-shot part segmentation · PartNet-E (Overall 45 categories) | mIoU: 65.9 -> 72.8 | Helped |
| Use SegGraph with CLIP and SAM instead of PointCLIPv2 | ShapeNetPart few-shot part segmentation · ShapeNetPart (Overall 16) | mIoU: 57.0 -> 62.6 | Helped |
| Add SAM-derived 3D segments to average-pooled DINOv2 features (ablation) | PartNet-E vanilla feature-pooling baseline · PartNet-E (Overall 45) | mIoU: 65.2 -> 70.5 | Helped |
| Add segment encoding to the SAM-segment model (ablation) | PartNet-E SAM-segment model · PartNet-E (Overall 45) | mIoU: 70.5 -> 70.7 | No clear effect |
| Add viewing quality-aware unpooling to the SAM-segment model (ablation) | PartNet-E SAM-segment model · PartNet-E (Overall 45) | mIoU: 70.5 -> 70.5 | No clear effect |
| Add segment encoding and viewing quality-aware unpooling together (ablation) | PartNet-E SAM-segment model · PartNet-E (Overall 45) | mIoU: +0.5% | Helped |
| Add overlap and adjacency graph edges (ablation) | PartNet-E SAM-segment model without segment encoding or quality-aware unpooling · PartNet-E (Overall 45) | mIoU: +0.9% | Helped |
| Add overlap graph edges to segment encoding and quality-aware unpooling (ablation) | PartNet-E SegGraph variant · PartNet-E (Overall 45) | mIoU: 1.3% | Helped |
| Add adjacency graph edges to segment encoding and quality-aware unpooling (ablation) | PartNet-E SegGraph variant · PartNet-E (Overall 45) | mIoU: 1.2% | Helped |
| Use overlap and adjacency graph edges together with segment encoding and quality-aware unpooling (ablation) | PartNet-E SegGraph variant · PartNet-E (Overall 45) | mIoU: 1.8% | Helped |
| Replace the MLP with graph propagation (ablation) | PartNet-E feature extractor variants · PartNet-E (Overall 45) | mIoU | Helped |
| Replace SAM with FastSAM (ablation) | SegGraph · PartNet-E | mIoU: 72.8 → 70.9 | Hurt |
Surprising findings
- Replacing DINOv2 with GLIP features yields only a small drop in performance (70.4% vs 72.8%), and the MLP-to-graph replacement helps consistently across all foundation model features.
- Applying segment encoding or viewing-quality unpooling alone gives little or no gain; they help mainly when combined with graph edges.
Experimental setup
Datasets: PartNet-E ShapeNetPart
Benchmarks: PartNet-E ShapeNetPart
Metrics: mIoU
Baselines: PointNet++ SoftGroup PointNext PartSLIP PartSLIP++ PartSTAD 3-By-2 PartDistill PartField PointCLIP PointCLIPv2
Builds on: PartSLIP PartSLIP++ PartDistill GATv2 SAM
Closest papers by meaning
- Bridge the Points: Graph-based Few-shot Segment Anything Semantically (NeurIPS 2024 · similarity 0.89)
- BFANet: Revisiting 3D Semantic Segmentation with Boundary Feature Analysis (CVPR 2025 · similarity 0.88)
- No Time to Train: Empowering Non-Parametric Networks for Few-shot 3D Scene Segmentation (CVPR 2024 · similarity 0.87)
- OnlineAnySeg: Online Zero-Shot 3D Segmentation by Visual Foundation Model Guided 2D Mask Merging (CVPR 2025 · similarity 0.87)
- Geometry-Guided Semantic Reconstruction for 3D Instance Segmentation (NeurIPS 2026 · similarity 0.86)
- Topograph: An Efficient Graph-Based Framework for Strictly Topology Preserving Image Segmentation (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.