Research map / Vision-language and multimodal / Cross-modal learning
Hyperbolic Gramian Volumes for Multimodal Alignment
Saiyang Na, Feng Jiang, Qifeng Zhou, Wenliang Zhong, Thao M. Dang, Yuzhi Guo, Hehuan Ma, Chunyuan Li, Weizhi An, Junzhou Huang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Euclidean Gramian volume with hybrid Euclidean-hyperbolic volume | GRAM on VAST backbone · MSR-VTT zero-shot | T2V Recall@1: 54.8 -> 56.6 | Improved |
| use pure hyperbolic volume (alpha = 0) instead of Euclidean GRAM (ablation) | GRAM · MSR-VTT zero-shot | T2V Recall@1: 54.8 -> 54.8 | No clear effect |
| use pure hyperbolic volume instead of Euclidean GRAM (ablation) | GRAM · DiDeMo and VATEX zero-shot | T2V Recall@1: DiDeMo 49.8 -> 49.1; VATEX 77.0 -> 76.7 | Hurt |
| use pure hyperbolic volume instead of Euclidean GRAM (ablation) | GRAM · ActivityNet zero-shot | T2V Recall@1: 56.2 -> 57.0 | Improved |
| add learnable hybrid mixing (alpha = 0.5148) over pure hyperbolic (ablation) | HyperGRAM · MSR-VTT zero-shot | T2V Recall@1: 0.548 -> 0.566 | Improved |
| switch from Euclidean volume to hyperbolic Lorentzian volume: variance of volumes | Gramian volume · MSR-VTT volume statistics | volume std: 0.005 -> 0.12 | Improved |
| set curvature c = 1.0 versus near-Euclidean c = 0.05 (ablation) | HyperGRAM · MSR-VTT zero-shot | T2V Recall@1: 0.551 -> 0.566 | Improved |
| raise curvature to c = 9.0 (ablation) | HyperGRAM · MSR-VTT zero-shot | T2V Recall@1: 0.566 -> 0.564 | No clear effect |
| add audio modality to hybrid volume (ablation) | HyperGRAM · MSR-VTT zero-shot | T2V Recall@1: 0.532 -> 0.548 | Improved |
| add subtitle modality (TVAS vs TVA) (ablation) | HyperGRAM · MSR-VTT zero-shot | T2V Recall@1: 0.548 -> 0.566 | Improved |
| vary alpha across [0.3, 0.7] around the learned value (ablation) | HyperGRAM · four video-text benchmarks | T2V Recall@1 | No clear effect |
| use hybrid geometry instead of either pure space (ablation) | HyperGRAM · MSR-VTT | T2V Recall@1: +1.3-1.8% | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- VAST — VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset (NeurIPS 2023) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Accept the Modality Gap: An Exploration in the Hyperbolic Space (CVPR 2024 · similarity 0.88)
- HyperET: Efficient Training in Hyperbolic Space for Multi-modal Large Language Models (NeurIPS 2025 · similarity 0.87)
- Gramian Multimodal Representation Learning and Alignment (ICLR 2025 · similarity 0.87)
- Learning Along the Arrow of Time: Hyperbolic Geometry for Backward-Compatible Representation Learning (ICML 2025 · similarity 0.86)
- HELM: Hyperbolic Large Language Models via Mixture-of-Curvature Experts (NeurIPS 2025 · similarity 0.85)
- CovMatch: Cross-Covariance Guided Multimodal Dataset Distillation with Trainable Text Encoder (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
