Research map / Point clouds / Point cloud classification
X-Trans2Cap: Cross-Modal Knowledge Transfer Using Transformer for 3D Dense Captioning
Zhihao Yuan, Xu Yan, Yinghong Liao, Yao Guo, Guanbin Li, Shuguang Cui, Zhen Li
Check your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with a multimodal teacher, 3D-only student, CMF, and feature alignment | TransCap · ScanRefer Oracle DC | CIDEr: 75.75 -> 87.09 | Improved |
| Train with a multimodal teacher, 3D-only student, CMF, and feature alignment | TransCap · Nr3D Oracle DC | CIDEr: 70.60 -> 80.02 | Improved |
| Use cross-modal knowledge transfer | TransCap · ScanRefer Scan DC | CIDEr@0.25: 55.36 -> 58.81 | Improved |
| Add CIDEr-D reward loss | X-Trans2Cap · ScanRefer Oracle DC | CIDEr: 87.09 -> 97.17 | Improved |
| Initialize with a separately pre-trained teacher instead of joint teacher-student training | X-Trans2Cap · ScanRefer Oracle DC | CIDEr: 87.09 -> 79.41 | Hurt |
| Add feature alignment loss (ablation) | X-Trans2Cap · ScanRefer Oracle DC | CIDEr: 79.58 -> 87.09 | Improved |
| Add cross-modal fusion module (ablation) | X-Trans2Cap · ScanRefer Oracle DC | CIDEr: 80.54 -> 87.09 | Improved |
| Use the standard CMF rather than concatenation (ablation) | X-Trans2Cap · ScanRefer Oracle DC | CIDEr: 79.87 -> 87.09 | Improved |
| Add random masking to CMF (ablation) | X-Trans2Cap · ScanRefer Oracle DC | CIDEr: 85.36 -> 87.09 | Improved |
| Use standard CMF rather than attention-based fusion (ablation) | X-Trans2Cap · ScanRefer Oracle DC | CIDEr: 80.85 -> 87.09 | Improved |
| Add object class to the student input (ablation) | X-Trans2Cap · ScanRefer Oracle DC | CIDEr: 70.41 -> 87.09 | Improved |
| Add 3D bounding box to the student input (ablation) | X-Trans2Cap · ScanRefer Oracle DC | CIDEr: 82.99 -> 87.09 | Improved |
Later papers that compared with this method
1 later accepted paper reports X-Trans2Cap* as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- TOD3Cap: Towards 3D Dense Captioning in Outdoor Scenes (ECCV 2024 · better on 1 of 1 shared result)
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- Scan2Cap — Scan2Cap: Context-Aware Dense Captioning in RGB-D Scans (CVPR 2021) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Benchmarking Large Vision-Language Models via Directed Scene Graph for Comprehensive Image Captioning (CVPR 2025 · similarity 0.86)
- CLIP-Driven Open-Vocabulary 3D Scene Graph Generation via Cross-Modality Contrastive Learning (CVPR 2024 · similarity 0.85)
- UniGS: Unified Language-Image-3D Pretraining with Gaussian Splatting (ICLR 2025 · similarity 0.85)
- X4D-SceneFormer: Enhanced Scene Understanding on 4D Point Cloud Videos through Cross-Modal Knowledge Transfer (AAAI 2024 · similarity 0.85)
- MMScan: A Multi-Modal 3D Scene Dataset with Hierarchical Grounded Language Annotations (NeurIPS 2024 · similarity 0.85)
- More Text, Less Point: Towards 3D Data-Efficient Point-Language Understanding (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
