Research map / 3D vision / NeRF and radiance fields
Neural Processing of Tri-Plane Hybrid Neural Fields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Processes tri-plane hybrid neural fields directly with standard architectures (Transformer invariant to channel order) to classify and segment 3D objects represented as UDF, SDF, OF and RF fields, outperforming MLP-based neural field processing methods.
Problem
Directly processing neural fields for 3D tasks like classification and part segmentation has been limited: shared-network methods sacrifice reconstruction, while single-MLP methods suffer from high-dimensional weight spaces, weight symmetries and initialization sensitivity, yielding weak task perfor
Approach
Process only the discrete tri-plane feature maps of per-object hybrid neural fields with standard architectures, and make them robust to random initialization by using a Transformer encoder without positional encoding over channel tokens, which is invariant to channel permutations.
Key results
- Tri-plane Transformer reaches 86.8% accuracy on Manifold40 SDF classification and 94.1% on ShapeNet10 UDF, versus 4.2 for MLP baseline and 47.9 for DWSNet on Manifold40 (single-MLP methods).
- On ShapeNetPart UDF segmentation, the method achieves 84.2 instance mIoU vs 64.2 for inr2vec, approaching PointNet++ (84.9).
- Tri-plane reconstructions reach 0.18 mm Chamfer Distance on Manifold40 meshes with 64K parameters, versus 0.26 mm for inr2vec with 800K parameters.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Process neural fields as tri-plane features rather than large MLPs | Neural field classification · ModelNet40, ShapeNet10, ScanNet10 (UDF); Manifold40 (SDF); ShapeNet10 (OF); ShapeNetRender (RF) | accuracy: 10.6 -> 87.0; 42.0 -> 94.1; 40.9 -> 69.1; 13.1 -> 86.8; 38.6 -> 91.8; 87.0 -> 92.6 | Helped |
| Process tri-plane features rather than use shared-network Functa | Neural field classification · ScanNet10 (UDF) | accuracy: 56.4 -> 69.1 | Helped |
| Use tri-plane features rather than MLP weights | ShapeNetPart segmentation · ShapeNetPart (UDF) | instance mIoU: 64.2 -> 84.2 | Helped |
| Use tri-plane features rather than MLP weights | ShapeNetPart segmentation · ShapeNetPart (UDF) | class mIoU: 64.5 -> 81.3 | Helped |
| Replace Transformer with flattened-input MLP (ablation) | Tri-plane classifier · ModelNet40 (UDF) | accuracy: 87.0 -> 41.6 | Hurt |
| Replace Transformer with CNN (ablation) | Tri-plane classifier · ModelNet40 (UDF) | accuracy: 87.0 -> 82.2 | Hurt |
| Replace Transformer with channel-order-invariant PointNet (ablation) | Tri-plane classifier · ModelNet40 (UDF) | accuracy: 87.0 -> 85.8 | Hurt |
| Use spatially invariant PointNet variant instead of channel-order-invariant processing (ablation) | Tri-plane classifier · ModelNet40 (UDF) | accuracy: 87.0 -> 32.3 | Hurt |
| Replace Transformer with CNN (ablation) | Tri-plane classifier · Manifold40 (SDF) | accuracy: 86.8 -> 82.5 | Hurt |
| Replace Transformer with PointNet (ablation) | Tri-plane classifier · Manifold40 (SDF) | accuracy: 86.8 -> 85.6 | Hurt |
| Shuffle tri-plane features across spatial dimensions (ablation) | Tri-plane SDF reconstruction · Manifold40 test (mesh from SDF) | CD (mm): 0.16mm -> 2.9mm | Hurt |
| Use tri-plane instead of voxel-grid hybrid representation | Hybrid-field reconstruction · Manifold40 test (mesh from SDF) | CD (mm): 0.19 -> 0.18 | Helped |
Surprising findings
- Classifying the MLPs of tri-plane fields directly yields poor accuracy (e.g., 4.3 for MLP on Manifold40), showing the discrete tri-plane features, not the MLP, carry the useful information.
- A simple MLP on flattened tri-planes achieves only 40.2 on Manifold40 SDF, indicating that channel-permutation invariance and spatial structure matter.
Limitations
- Tri-plane neural fields are specific to 3D world modeling; other hybrid fields and signals are not addressed.
- Channel-permutation alignment between independently initialized tri-planes was found by brute-force search only in a small C=8 setting.
- Universal classifier results are preliminary and only cover UDF, SDF and OF.
Experimental setup
Datasets: ModelNet40 ShapeNet10 ScanNet10 Manifold40 ShapeNetRender ShapeNetPart
Benchmarks: ModelNet40 ShapeNet10 ScanNet10 Manifold40 ShapeNetRender ShapeNetPart
Metrics: accuracy Chamfer Distance F-score instance mIoU class mIoU
Baselines: inr2vec NFN NFT DWSNet DeepSDF Functa PointNet PointNet++ DGCNN MeshWalker Conv3DNet ResNet50
Builds on: Functa inr2vec DWSNet NFN NFT
Closest papers by meaning
- DUDF: Differentiable Unsigned Distance Fields with Hyperbolic Scaling (CVPR 2024 · similarity 0.86)
- ResFields: Residual Neural Fields for Spatiotemporal Signals (ICLR 2024 · similarity 0.86)
- How to Train Neural Field Representations: A Comprehensive Study and Benchmark (CVPR 2024 · similarity 0.86)
- 3D-SLNR: A Super Lightweight Neural Representation for Large-scale 3D Mapping (CVPR 2025 · similarity 0.86)
- Grounding Continuous Representations in Geometry: Equivariant Neural Fields (ICLR 2025 · similarity 0.86)
- Accurate Differential Operators for Hybrid Neural Fields (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.