Research map / Point clouds / Point cloud classification
MESC-3D:Mining Effective Semantic Cues for 3D Reconstruction from a Single Image
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MESC-3D, a single-image point-cloud reconstruction method that uses a point-selection map (Effective Semantic Mining Module) and CLIP-based deep prompt tuning with learnable text prompts (3D Semantic Prior Learning Module) to reduce semantic entanglement and recover occluded geometry.
Problem
Single-image 3D point cloud reconstruction suffers because image semantic features are entangled when simply concatenated or added to point features, and occluded regions lack information from a single view.
Approach
A per-stage point-selection map masks features so each point selects needed semantics to predict scaling/translation, and a CLIP text encoder with deep prompt tuning and learnable text prompts is contrastively aligned with point cloud features to inject 3D geometric priors.
Key results
- On ShapeNet's 13 categories, MESC-3D achieves average CD 2.76 (x10^-2) and F-score@1% 0.974, beating the best competitor (BDM-B, CD 5.30, F 0.872) on every class.
- On Pix3D, MESC-3D reports CD 1.37 and F-score@1% 0.990, versus SDFusion CD 7.22 and 3DQD CD 8.65.
- Ablation on ShapeNet subset reduces average L2 CD from 5.68 (baseline) to 3.69 with ESM and 3.20 with 3DSPL added.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add ESM to the baseline (ablation) | MESC-3D · ShapeNet (13-category average) | Avg-CD (L2 CD×103): 5.68 -> 3.69 | Helped |
| Add 3DSPL to the model with ESM (ablation) | MESC-3D · ShapeNet (13-category average) | Avg-CD (L2 CD×103): 3.69 -> 3.20 | Helped |
| Use trainable ResNet18 rather than CLIP as the image encoder (ablation) | MESC-3D · Vessel | CD (L2 CD×103): 4.71 -> 2.74 | Helped |
| Fine-tune CLIP rather than use pre-trained CLIP (ablation) | MESC-3D · Vessel | CD (L2 CD×103): 4.71 -> 4.24 | Helped |
| Use DPT to fine-tune the text encoder with learnable prompts (ablation) | MESC-3D · Vessel | CD (L2 CD×103): 2.80 -> 2.63 | Helped |
| Concatenate 3D prior with fused features after MIT rather than use three-modal fusion (ablation) | MESC-3D · Vessel | CD (L2 CD×103): 3.26 -> 2.80 | Helped |
| Concatenate 3D prior with fused features after MIT rather than add it element-wise (ablation) | MESC-3D · Vessel | CD (L2 CD×103): 2.98 -> 2.80 | Helped |
| Place the category token in the middle rather than at the start of the prompt (ablation) | MESC-3D · ShapeNet | Avg-CD L2: 3.48 -> 3.20 | Helped |
| Place the category token in the middle rather than at the end of the prompt (ablation) | MESC-3D · ShapeNet | Avg-CD L2: 3.51 -> 3.20 | Helped |
| Use MESC-3D rather than 3DAttriFlow | single-image 3D reconstruction · ShapeNet (13-category average) | CD: 3.02 -> 2.76 | Helped |
| Use MESC-3D rather than SDFusion | single-image 3D reconstruction · Pix3D | CD: 7.22 -> 1.37 | Helped |
Limitations
- Zero-shot evaluation is limited to a few unseen Pix3D categories (bed, bookcase, desk, tool) with qualitative and limited quantitative reporting.
- Reported comparisons mix L1 and L2 Chamfer Distance scales across tables, making direct comparison difficult.
Experimental setup
Datasets: ShapeNet Pix3D
Benchmarks: ShapeNet Pix3D
Metrics: Chamfer Distance F-score@1%
Baselines: Point-E 3DAttriFlow PC² BDM-B SDFusion 3DQD
Builds on: CoOp CoCoOp Deep Prompt Tuning Multimodal Interlaced Transformer Point-MAE
Closest papers by meaning
- Know Your Neighbors: Improving Single-View Reconstruction via Spatial Vision-Language Reasoning (CVPR 2024 · similarity 0.89)
- Large Spatial Model: End-to-end Unposed Images to Semantic 3D (NeurIPS 2024 · similarity 0.88)
- ImageNet3D: Towards General-Purpose Object-Level 3D Understanding (NeurIPS 2024 · similarity 0.88)
- Tackling View-Dependent Semantics in 3D Language Gaussian Splatting (ICML 2025 · similarity 0.88)
- Learning Partonomic 3D Reconstruction from Image Collections (CVPR 2025 · similarity 0.87)
- SIU3R: Simultaneous Scene Understanding and 3D Reconstruction Beyond Feature Alignment (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.