Research map / Vision-language and multimodal / Cross-modal learning
Y2Seq2Seq: Cross-Modal Representation Learning for 3D Shape and Text by Joint Reconstruction and Prediction of View and Word Sequences
Zhizhong Han, Mingyang Shang, Xiyang Wang, Yu-Shen Liu, Matthias Zwicker
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use coupled branches with joint within-modality reconstruction and cross-modal prediction (ablation) | Y²Seq2Seq · Primitive S2T | RR@1: 69.87 -> 80.13 | Improved |
| Use coupled branches with joint within-modality reconstruction and cross-modal prediction (ablation) | Y²Seq2Seq · ShapeNet S2T | RR@1: 1.74 -> 1.88 | Improved |
| Add the class-level classification constraint (ablation) | Y²Seq2Seq · Primitive S2T | RR@1: 80.13 -> 83.07 | Improved |
| Add the triplet constraint after classification (ablation) | Y²Seq2Seq · Primitive S2T | RR@1: 83.07 -> 88.53 | Improved |
| Add the joint embedding constraint after classification and triplet constraints (ablation) | Y²Seq2Seq · Primitive S2T | RR@1: 88.53 -> 94.13 | Improved |
| Add all hierarchical constraints (ablation) | Y²Seq2Seq · ShapeNet S2T | RR@1: 1.88 -> 6.77 | Improved |
| Increase voxel resolution from 32³ to 128³ (ablation) | Y²Seq2Seq · ShapeNet S2T | RR@1: 6.77 -> 7.64 | Improved |
| Increase voxel resolution from 32³ to 64³ (ablation) | Y²Seq2Seq · ShapeNet T2S | RR@1: 2.93 -> 2.37 | Hurt |
| Increase voxel resolution from 32³ to 128³ (ablation) | Y²Seq2Seq · ShapeNet T2S | RR@5: 9.23 -> 9.82 | Improved |
| Generate captions rather than retrieve the nearest description (ablation) | Y²Seq2Seq · Primitive shape captioning | METEOR: 0.70 -> 0.54 | Hurt |
| Generate captions rather than retrieve the nearest description (ablation) | Y²Seq2Seq · ShapeNet shape captioning | METEOR: 0.22 -> 0.29 | Improved |
Closest papers by meaning
- HOPE: A Hierarchical Perspective for Semi-Supervised 2D-3D Cross-Modal Retrieval (TPAMI 2024 · similarity 0.88)
- TAMM: TriAdapter Multi-Modal Learning for 3D Shape Understanding (CVPR 2024 · similarity 0.87)
- Escaping Plato's Cave: Towards the Alignment of 3D and Text Latent Spaces (CVPR 2025 · similarity 0.87)
- Connect, Collapse, Corrupt: Learning Cross-Modal Tasks with Uni-Modal Data (ICLR 2024 · similarity 0.87)
- CL2CM: Improving Cross-Lingual Cross-Modal Retrieval via Cross-Lingual Knowledge Transfer (AAAI 2024 · similarity 0.86)
- Same Concept, Different Directions: Cross-Modal Feature Heterogeneity in Sparse Autoencoders (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
