Research map / Point clouds / Point cloud classification
Point-Cache: Test-time Dynamic and Hierarchical Cache for Robust and Generalizable Point Cloud Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Point-Cache, a training-free hierarchical (global + local part) test-time cache built from online test samples that adapts frozen large multimodal 3D models' zero-shot predictions for robust and open-vocabulary point cloud recognition.
Problem
Point cloud recognition models degrade under distribution shifts and corruptions at test time, and existing test-time adaptation methods depend on training data and can only recognize training classes.
Approach
A hierarchical key-value cache built solely from online test samples: global features plus K-Means-clustered local part features from the last transformer layer, with entropy-based selective update, whose affinity-weighted pseudo-label votes are added to frozen zero-shot logits of large 3D models.
Key results
- Hierarchical cache raises ULIP's average accuracy over 7 ModelNet-C corruptions from 47.52% to 53.70%, and ULIP-2's from 57.95% to 62.79%.
- On ScanObjectNN hardest split, hierarchical cache lifts ULIP from 27.20% to 51.80% and ULIP-2 from 36.22% to 54.98%.
- Uni3D with the hierarchical cache reaches 89.18% on ModelNet40 without training, matching fully supervised PointNet (89.20%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add global cache to zero-shot model | ULIP, ULIP-2, OpenShape, Uni3D · ModelNet-C (avg over 7 corruptions) | accuracy (%): +5.04 (ULIP), +3.23 (ULIP-2), +2.94 (OpenShape), +3.86 (Uni3D) | Helped |
| add hierarchical cache (global + local) to zero-shot model | ULIP, ULIP-2, OpenShape, Uni3D · ModelNet-C (avg over 7 corruptions) | accuracy (%): +6.18 (ULIP), +4.84 (ULIP-2), +3.10 (OpenShape vs. baseline 73.49), +5.36 (Uni3D) | Helped |
| add hierarchical cache to zero-shot model | OpenShape · ModelNet40 clean | accuracy (%): 84.56 to 84.04 | Hurt |
| add hierarchical cache to zero-shot model | ULIP · ModelNet40 clean | accuracy (%): 56.16 to 64.22 | Helped |
| add global cache | ULIP-2 · Objaverse-LVIS | accuracy (%): 30.26 to 32.65 | Helped |
| add hierarchical cache to ULIP-2 vs global cache | ULIP-2 · Objaverse-LVIS | accuracy (%): 32.65 (global) vs 32.36 (hierarchical) | Mixed |
| add hierarchical cache | ULIP, ULIP-2, Uni3D · Average over ModelNet40, S-PB_T50_RS, Omni3D, O-LVIS | accuracy (%): ULIP +6.31, ULIP-2 +5.51 (average over 6 scores as stated) | Helped |
| add hierarchical cache to ULIP-2 | ULIP-2 · Throughput on ModelNet40 | throughput (t/s): 11.18 to 11.11 | Hurt |
| global cache size K=3 vs other K values (ablation) | Uni3D + hierarchical cache · Average over 4 datasets (ModelNet40, S-PB_T50_RS, Omni3D, O-LVIS) | accuracy (%): best average 63.98% at K=3 | Mixed |
| increase number of local parts per object beyond 3 (ablation) | Uni3D + hierarchical cache · S-PB_T50_RS, Omni3D, O-LVIS | accuracy (%) | Hurt |
| increase balance factors alpha_g = alpha_l from 0.5 to 4.0 (ablation) | Uni3D + hierarchical cache · S-PB_T50_RS and Omni3D | accuracy (%) | Helped |
| increase sharpness beta beyond optimal (3.0) (ablation) | Uni3D + hierarchical cache · Average over 4 datasets | accuracy (%) | Mixed |
Surprising findings
- Adding the cache sometimes slightly reduces clean ModelNet accuracy for OpenShape (84.56% zero-shot vs 84.04% hierarchical), an exception to the general gains.
Limitations
- Gains on ScanObjectNN-C and some corruptions are uneven; the cache can slightly hurt performance on some settings (e.g., clean ModelNet for OpenShape).
- Hyperparameters (K, m, alpha, beta) were tuned via ablations on the evaluation benchmarks themselves.
- Local parts are fuzzy, defined as transformer patch outputs clustered by K-Means rather than semantic parts.
Experimental setup
Datasets: ModelNet-C ScanObjectNN-C ModelNet40 ScanObjectNN OmniObject3D Objaverse-LVIS
Benchmarks: ModelNet-C ScanObjectNN-C ModelNet40 ScanObjectNN OmniObject3D Objaverse-LVIS
Metrics: accuracy throughput memory usage
Baselines: ULIP ULIP-2 OpenShape Uni3D MATE BFTT3D CloudFixer
Builds on: Tip-Adapter ULIP OpenShape
Closest papers by meaning
- Point2Real: Bridging the Gap between Point Cloud and Realistic Image for Open-World 3D Recognition (AAAI 2024 · similarity 0.88)
- Occlusion-aware Text-Image-Point Cloud Pretraining for Open-World 3D Object Recognition (CVPR 2025 · similarity 0.88)
- Sonata: Self-Supervised Learning of Reliable Point Representations (CVPR 2025 · similarity 0.88)
- Point-PRC: A Prompt Learning Based Regulation Framework for Generalizable Point Cloud Analysis (NeurIPS 2024 · similarity 0.87)
- SMART-PC: Skeletal Model Adaptation for Robust Test-Time Training in Point Clouds (ICML 2025 · similarity 0.87)
- PointMamba: A Simple State Space Model for Point Cloud Analysis (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.