Research map / 3D vision / NeRF and radiance fields
OpenHype: Hyperbolic Embeddings for Hierarchical Open-Vocabulary Radiance Fields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OpenHype, which learns a single hyperbolic (Lorentz) latent feature field in a NeRF so that open-vocabulary hierarchical queries can be answered by traversing geodesics to the origin in one rendering pass.
Problem
Open-vocabulary 3D segmentation from radiance fields handles hierarchical (object and part) queries poorly: existing methods need multiple rendering passes or fixed discrete hierarchy levels.
Approach
Train a hyperbolic autoencoder with contrastive distance and exterior-angle losses to embed SAM-derived object hierarchies (CLIP features) in the Lorentz model, supervise a NeRF with geodesic-distance losses on extrapolated hyperbolic features, and at query time sample points along the geodesic to the origin and aggregate CLIP relevancy scores with softmax weighting.
Key results
- On the adapted Search3D benchmark, OpenHype reaches 35.1 average IoU and 69.5 average accuracy, versus 25.8 IoU for LangSplat (best baseline).
- On the LERF dataset, OpenHype averages 54.6 mIoU, slightly above N2F2 (54.4) and LangSplat (51.4).
- Ablations show softmax path aggregation gives 30.9 overall vs 29.8 for max and 24.3 for sum; the paper reports an oracle LangSplat that is still below OpenHype.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace flat baseline representations with continuous hyperbolic hierarchy embeddings | OpenHype · Search3D ScanNet++ Avg. | IoU: 25.8 -> 35.1 | Helped |
| replace flat baseline representations with continuous hyperbolic hierarchy embeddings | OpenHype · LERF Avg. | mIoU: 54.4 -> 54.6 | Helped |
| add hyperbolic features rather than CLIP embeddings for NeRF supervision (ablation) | OpenHype · Search3D five-scene ablation | IoU: 10.9 -> 26.4 | Helped |
| supervise directly in hyperbolic space instead of tangent space (ablation) | OpenHype · Search3D five-scene ablation | IoU: 24.7 -> 26.4 | Helped |
| add feature extrapolation toward the hyperbolic-space boundary (ablation) | OpenHype · Search3D five-scene ablation | IoU: 26.4 -> 30.9 | Helped |
| add negative prompts (ablation) | OpenHype · Search3D five-scene ablation | IoU: 28.7 -> 30.9 | Helped |
| combine positive and negative prompts at every geodesic step rather than after aggregation (ablation) | OpenHype · Search3D five-scene ablation | IoU: 30.7 -> 30.9 | No clear effect |
| add angle loss to distance loss (ablation) | OpenHype auto-encoder · Search3D five-scene ablation | IoU: 28.8 -> 30.9 | Helped |
| use angle loss without distance loss (ablation) | OpenHype auto-encoder · Search3D five-scene ablation | IoU | Hurt |
| replace softmax path aggregation with maximum aggregation (ablation) | OpenHype · Search3D five-scene ablation | IoU: 30.9 -> 29.8 | Hurt |
| replace softmax path aggregation with mean aggregation (ablation) | OpenHype · Search3D five-scene ablation | IoU: 30.9 -> 30.3 | Hurt |
| replace softmax path aggregation with sum aggregation (ablation) | OpenHype · Search3D five-scene ablation | IoU: 30.9 -> 24.3 | Hurt |
Surprising findings
- Using the exterior-angle loss alone causes the autoencoder to collapse (NaN results), so it only works combined with the distance loss.
- A pretrained hyperbolic CLIP (MERU) substituted for the autoencoder performed poorly.
Limitations
- Quality depends on the 2D segmentation model; object parts not segmented in any view cannot be represented.
- Inconsistent CLIP features for visually similar parts introduce multi-view supervision noise.
- Some noise appears along geodesic paths between rendered features and the origin.
Experimental setup
Datasets: Search3D (ScanNet++ subset) LERF dataset
Benchmarks: Search3D (ScanNet++ subset, adapted) LERF dataset
Metrics: mIoU localization accuracy
Baselines: LERF OpenNeRF LangSplat N2F2 LSeg OpenSeg Hi-LSplat
Builds on: NeRF LERF Nerfacto Semantic SAM MERU
Closest papers by meaning
- Rethinking Open-Vocabulary Segmentation of Radiance Fields in 3D Space (AAAI 2025 · similarity 0.88)
- Latent Radiance Fields with 3D-aware 2D Representations (ICLR 2025 · similarity 0.87)
- GaussianCube: A Structured and Explicit Radiance Representation for 3D Generative Modeling (NeurIPS 2024 · similarity 0.87)
- HyRF: Hybrid Radiance Fields for Memory-efficient and High-quality Novel View Synthesis (NeurIPS 2025 · similarity 0.86)
- LiveScene: Language Embedding Interactive Radiance Fields for Physical Scene Control and Rendering (NeurIPS 2024 · similarity 0.86)
- BLiRF: Bandlimited Radiance Fields for Dynamic Scene Modeling (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.