Research map / Point clouds / Point cloud classification
Point2Real: Bridging the Gap between Point Cloud and Realistic Image for Open-World 3D Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Point2Real converts point clouds into realistic rendered images (ball-pivoting meshing, texture candidate rendering with CLIP-based selection, depth-surface multi-view weighting) so frozen CLIP can classify 3D objects training-free, with a dual-path adapter for few-shot.
Problem
Zero-shot and few-shot 3D point cloud classification with 2D vision-language models like CLIP suffers from a large domain gap between discrete point clouds and realistic images (shape, texture, viewpoint).
Approach
Instead of projecting points to depth maps, reconstruct meshes with an iterative ball-pivoting algorithm, render them with ray tracing and candidate textures selected by CLIP confidence, and weight viewpoints by depth-surface area, all without 3D training.
Key results
- Zero-shot accuracy of 79.5% on ModelNet10, surpassing PointCLIP V2 (73.1%) and ReCon (75.6%) and exceeding all training-free methods.
- Zero-shot accuracy of 65.8% on ModelNet40, +1.6% over PointCLIP V2 (64.2%).
- One-shot ModelNet40 few-shot accuracy of 67.5%, +14.9% over PointCLIP (52.6%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Point2Real instead of ReCon | zero-shot 3D classification · ModelNet10 | accuracy: 75.6 -> 79.5 | Helped |
| Use Point2Real instead of ReCon | zero-shot 3D classification · ModelNet40 | accuracy: 61.7 -> 65.8 | Helped |
| Use Point2Real instead of PointCLIP V2 | zero-shot 3D classification · ScanObjectNN OBJ ONLY | accuracy: 50.1 -> 57.7 | Helped |
| Use Point2Real instead of PointCLIP V2 | zero-shot 3D classification · ScanObjectNN OBJ BG | accuracy: 41.2 -> 53.0 | Helped |
| Use Point2Real instead of ULIP | zero-shot 3D classification · ScanObjectNN PB T50 RS | accuracy: 49.9 -> 38.1 | Hurt |
| Add mesh reconstruction in place of depth projection (ablation) | Point2Real · ModelNet10 zero-shot | accuracy: 73.1 -> 75.9 | Helped |
| Add mesh reconstruction in place of depth projection (ablation) | Point2Real · ModelNet40 zero-shot | accuracy: 64.2 -> 60.3 | Hurt |
| Add texture rendering to mesh reconstruction (ablation) | Point2Real · ModelNet10 zero-shot | accuracy: 75.9 -> 78.4 | Helped |
| Add texture rendering to mesh reconstruction (ablation) | Point2Real · ModelNet40 zero-shot | accuracy: 60.3 -> 65.3 | Helped |
| Add the multi-view adapter (ablation) | Point2Real · ModelNet10 zero-shot | accuracy: 78.4 -> 79.5 | Helped |
| Add the multi-view adapter (ablation) | Point2Real · ModelNet40 zero-shot | accuracy: 65.3 -> 65.8 | Helped |
| Use five-iteration Ball-Pivoting rather than one iteration (ablation) | Point2Real · ModelNet40 zero-shot | accuracy: 78.6 -> 79.5 | Helped |
Surprising findings
- Replacing textures with plain white coloration reduces ModelNet10/40 zero-shot accuracy, showing CLIP is sensitive to texture cues.
- Poisson surface reconstruction drops ModelNet40 zero-shot accuracy to 56.3%, far below iterative ball pivoting (79.5% on ModelNet10 setting), indicating sensitivity to point density.
Limitations
- Blender-based rendering is slow (about 2.9 s per sample), limiting real-time use.
- Multi-view adapter gives small gains on standard datasets because most viewpoints are already distinguishable.
- Few-shot gains shrink as the number of shots grows.
Experimental setup
Datasets: ModelNet10 ModelNet40 ScanObjectNN
Benchmarks: ModelNet10 ModelNet40 ScanObjectNN
Metrics: accuracy inference time
Baselines: PointCLIP PointCLIP V2 CLIP2Point ReCon ULIP Cheraghian
Builds on: PointCLIP PointCLIP V2 CLIP Ball-Pivoting algorithm
Closest papers by meaning
- Occlusion-aware Text-Image-Point Cloud Pretraining for Open-World 3D Object Recognition (CVPR 2025 · similarity 0.90)
- Geometrically-driven Aggregation for Zero-shot 3D Point Cloud Understanding (CVPR 2024 · similarity 0.89)
- OpenDlign: Open-World Point Cloud Understanding with Depth-Aligned Images (NeurIPS 2024 · similarity 0.88)
- Duoduo CLIP: Efficient 3D Understanding with Multi-View Images (ICLR 2025 · similarity 0.88)
- UniPre3D: Unified Pre-training of 3D Point Cloud Models with Cross-Modal Gaussian Splatting (CVPR 2025 · similarity 0.88)
- Multi-View Representation is What You Need for Point-Cloud Pre-Training (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.