Research map / 3D vision / 3D generation
Improving 2D Feature Representations by 3D-Aware Fine-Tuning
Yuanwen Yue, Anurag Das, Francis Engelmann, Siyu Tang, Jan Eric Lenssen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| concatenate original DINOv2 features with 3D-aware fine-tuned features instead of adding them (ablation) | FiT3D · NYUv2 segmentation | mIoU: 66.0 -> 67.5 | Improved |
| concatenate instead of linear fusion (ablation) | FiT3D · NYUv2 segmentation | mIoU: 66.39 -> 67.5 | Improved |
| increase per-Gaussian feature dimension from 32 to 64 (ablation) | feature Gaussians · NYUv2 segmentation | mIoU: 67.15 -> 67.5 | Improved |
| fine-tune for 2 or 3 epochs instead of 1 (ablation) | FiT3D · NYUv2 segmentation | mIoU: 67.5 -> 67.25 | No clear effect |
| fine-tune DINOv2 directly instead of frozen DINOv2 plus adapter (ablation) | FiT3D · Pascal VOC segmentation | mIoU: 82.02 -> 82.35 | No clear effect |
| add fine-tuned 3D-aware features to DINOv2 (vs just duplicating original features) (ablation) | linear probing · ScanNet++ segmentation | mIoU: 30.31 -> 32.76 | Improved |
| apply 3D-aware fine-tuning to DINOv2 ViT-S | DINOv2 · ScanNet++ segmentation | mIoU: 30.19 -> 32.76 | Improved |
| apply 3D-aware fine-tuning (trained only on ScanNet++) to out-of-domain depth | DINOv2 · KITTI depth | RMSE: 3.03 -> 2.91 | Improved |
| apply 3D-aware fine-tuning to other backbones (CLIP, MAE, DeiT-III, DINOv2-reg) | 2D vision models · ScanNet++ segmentation | mIoU: MAE 17.19 -> 20.27 | Improved |
| apply 3D-aware fine-tuning to DINOv2 | DINOv2 · ImageNet-1K classification | accuracy: 80.02 -> 80.00 | No clear effect |
| use ViT-B instead of ViT-S for 3D-aware fine-tuning | DINOv2 · ScanNet++ segmentation | mIoU: 32.72 -> 36.19 | Improved |
| use DPT head instead of linear probe with 3D-aware features | DINOv2 · ScanNet++ depth | RMSE: 0.3027 -> 0.2830 | Improved |
Closest papers by meaning
- Probing the 3D Awareness of Visual Foundation Models (CVPR 2024 · similarity 0.90)
- Feat2GS: Probing Visual Foundation Models with Gaussian Splatting (CVPR 2025 · similarity 0.88)
- Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Understanding (NeurIPS 2024 · similarity 0.88)
- Tackling View-Dependent Semantics in 3D Language Gaussian Splatting (ICML 2025 · similarity 0.87)
- Cross-Modal and Uncertainty-Aware Agglomeration for Open-Vocabulary 3D Scene Understanding (CVPR 2025 · similarity 0.87)
- SceneFun3D: Fine-Grained Functionality and Affordance Understanding in 3D Scenes (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
