Research map / Point clouds / Point cloud classification
PointAD: Comprehending 3D Anomalies from Points and Pixels for Zero-shot 3D Anomaly Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PointAD, a CLIP-based zero-shot 3D anomaly detection framework that renders point clouds into multi-view 2D images, projects features back to 3D, and learns normality/abnormality text prompts with hybrid point and pixel losses, extendable to RGB fusion without retraining.
Problem
Zero-shot 3D anomaly detection and segmentation on unseen point-cloud objects when no target normal training samples are available (e.g., privacy or new products), a setting unexplored before.
Approach
Render point clouds into multi-view 2D renderings encoded by frozen CLIP, project pixel features back to 3D points via visibility-masked rendering correspondence, and optimize learnable normal/abnormal text prompts jointly with MIL-based 3D losses and MTL-based 2D losses (hybrid representation learning); RGB features can be fused at test time without retraining.
Key results
- On MVTec3D-AD zero-shot 3D detection, PointAD improves I-AUROC from 61.2% (CLIP + Rendering) to 82.0% and AP from 85.8% to 94.2%.
- Multimodal (RGB + point) zero-shot detection raises MVTec3D-AD I-AUROC to 86.9% and P-AUROC to 97.2%.
- Cross-dataset generalization shows nearly no decay relative to the one-vs-rest setting on Eyecandies and Real3D-AD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace CLIP + Rendering with PointAD hybrid representation learning | CLIP + Rendering on MVTec3D-AD ZS 3D · MVTec3D-AD, ZS 3D | I-AUROC: 61.2% to 82.0% | Helped |
| Replace Cheraghian with PointAD | Cheraghian on MVTec3D-AD ZS 3D segmentation · MVTec3D-AD, ZS 3D | AUPRO: 57.0% to 84.4% | Helped |
| Integrate RGB information at test time | PointAD on MVTec3D-AD · MVTec3D-AD, ZS M3D | I-AUROC: 82.0% to 86.9% | Helped |
| Integrate RGB information at test time | PointAD on MVTec3D-AD · MVTec3D-AD, ZS M3D | AUPRO: 84.4% to 90.2% | Helped |
| Add 2D glocal loss (ablation) | PointAD multimodal segmentation · MVTec3D-AD, ZS M3D | P-AUROC: 92.9% to 97.2% | Helped |
| Add 2D glocal loss (ablation) | PointAD multimodal segmentation · MVTec3D-AD, ZS M3D | AUPRO: 84.4% to 90.2% | Helped |
| Increase auxiliary training categories from one to three (ablation) | PointAD point detection · MVTec3D-AD | P-AUROC: 95.5%, 96.1%, to 96.3% | Helped |
| Increase auxiliary training categories from one to three (ablation) | PointAD multimodal detection · MVTec3D-AD | I-AUROC: 86.9%, 87.1%, to 88.0% | Helped |
| Increase rendering views from 9 to 11 (ablation) | PointAD global multimodal detection · MVTec3D-AD | I-AUROC: 87.4% to 86.4% | Hurt |
| Increase prompt length from 12 to 16 tokens (ablation) | PointAD multimodal detection · MVTec3D-AD | I-AUROC: 86.9 to 84.7 | Hurt |
| Occlude point clouds (ablation) | PointAD point detection · MVTec3D-AD, point detection with occlusions | Global I-AUROC: 82.0 to 73.3 | Hurt |
Surprising findings
- Without the 2D global loss, RGB score maps become noisy and multimodal fusion degrades.
- Increasing rendering views from 9 to 11 reduced multimodal global I-AUROC from 87.4% to 86.4%.
Limitations
- Uses fixed rendering angles across diverse objects, with no filtering of low-quality renderings.
- Direct multimodal fusion can yield unpromising score maps when one modality deviates severely from ground truth (failure cases).
Experimental setup
Datasets: MVTec3D-AD Eyecandies Real3D-AD
Benchmarks: MVTec3D-AD Eyecandies Real3D-AD
Metrics: I-AUROC AP P-AUROC AUPRO
Baselines: CLIP + Rendering PointCLIP V2 Cheraghian AnomalyCLIP PointAD-CoOp
Builds on: CLIP AnomalyCLIP CoOp PointCLIP V2
Closest papers by meaning
- Align3D-AD: Cross-Modal Feature Alignment and Dual-Prompt Learning for Zero-shot 3D Anomaly Detection (NeurIPS 2026 · similarity 0.90)
- PO3AD: Predicting Point Offsets toward Better 3D Point Cloud Anomaly Detection (CVPR 2025 · similarity 0.88)
- Odd-One-Out: Anomaly Detection by Comparing with Neighbors (CVPR 2025 · similarity 0.87)
- Looking 3D: Anomaly Detection with 2D-3D Alignment (CVPR 2024 · similarity 0.87)
- Towards Scalable 3D Anomaly Detection and Localization: A Benchmark via 3D Anomaly Synthesis and A Self-Supervised Learning Network (CVPR 2024 · similarity 0.86)
- Towards Generalizable 3D Anomaly Detection via Relational Inconsistency Modeling (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.