Research map / Vision-language and multimodal / CLIP and zero-shot
ZeroKey: Point-Level Reasoning and Zero-Shot 3D Keypoint Detection from Large Language Models
Bingchen Gong, Diego Gomez, Abdullah Hamdi, Abdelrahman Eldesokey, Ahmed Abdelreheem, Peter Wonka, Maks Ovsjanikov
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add HDBSCAN clustering instead of direct averaging of back-projected points (ablation) | ZeroKey · KeypointNet | IoU | Improved |
| use Molmo instead of GPT-4o as the point-level MLLM backbone (ablation) | ZeroKey · KeypointNet | IoU: 20.73 -> 54.64 (IoU at threshold 0.10) | Improved |
| use point-specific prompts instead of one global prompt (ablation) | ZeroKey · KeypointNet | IoU | Improved |
| use 26 views instead of 6 views (ablation) | ZeroKey · KeypointNet | IoU: 80% performance with 6 views | No clear effect |
| ZeroKey versus CLIP-DINOiser baseline | ZeroKey · KeypointNet | IoU: 25.56 -> 54.64 (threshold 0.10) | Improved |
| ZeroKey versus RedCircle baseline | ZeroKey · KeypointNet | IoU: 18.50 -> 54.64 (threshold 0.10) | Improved |
| compare ZeroKey with the few-shot B2-3D using 26 views | ZeroKey · KeypointNet | IoU: 56.90 -> 54.64 (threshold 0.10) | Hurt |
| choose salient (describable) points instead of arbitrary points as targets | ZeroKey retrieval via Molmo description · Schelling points / KeypointNet | retrieval accuracy | Improved |
Closest papers by meaning
- Back to 3D: Few-Shot 3D Keypoint Detection with Back-Projected 2D Features (CVPR 2024 · similarity 0.91)
- Few-Shot Geometry-Aware Keypoint Localization (CVPR 2023 · similarity 0.89)
- Unsupervised 3D Structure Inference from Category-Specific Image Collections (CVPR 2024 · similarity 0.88)
- Geometrically-driven Aggregation for Zero-shot 3D Point Cloud Understanding (CVPR 2024 · similarity 0.88)
- Point2Real: Bridging the Gap between Point Cloud and Realistic Image for Open-World 3D Recognition (AAAI 2024 · similarity 0.87)
- ImOV3D: Learning Open Vocabulary Point Clouds 3D Object Detection from Only 2D Images (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
