Research map / Vision-language and multimodal / Cross-modal learning
Hyper-LLaVA: Hyperbolic Uncertainty-aware Modality-Balanced Routing for Multimodal Continual Instruction Tuning
Kunlun Xu, YanQin Zhang, Wenwen Qiang, Jiahuan Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Euclidean distance with Poincare distance (static fusion) (ablation) | Hyper-LLaVA routing · CoIN | MFN: 56.75 -> 62.67 | Improved |
| add BC-based adaptive modality fusion (Euclidean metric) (ablation) | Hyper-LLaVA routing · CoIN | MFN: 56.75 -> 66.42 | Improved |
| add Poincare metric on top of adaptive fusion (ablation) | Hyper-LLaVA routing · CoIN | MFN: 66.42 -> 66.62 | Improved |
| use adaptive fusion instead of static fusion (ablation) | Hyper-LLaVA routing · CoIN | MFN: 58.67 -> 66.62 | Improved |
| use adaptive fusion instead of static fusion (ablation) | Hyper-LLaVA routing · UCIT | MFN: 72.00 -> 72.81 | Improved |
| static fusion instead of instruction-only routing (ablation) | Hyper-LLaVA routing · CoIN | MFN: 61.78 -> 58.67 | Hurt |
| static fusion instead of unimodal routing (ablation) | Hyper-LLaVA routing · UCIT | MFN: 69.54 -> 72.00 | Improved |
| vary hyperbolic scaling coefficient gamma (ablation) | Hyper-LLaVA · CoIN | MFN | Mixed |
| Hyper-LLaVA vs HiDe-LLaVA hierarchical decoupling | Hyper-LLaVA · CoIN | MFN: 62.82 -> 66.47 | Improved |
| Hyper-LLaVA vs ModalPrompt on forgetting | Hyper-LLaVA · UCIT | BWT: -0.05 -> -5.22 | Hurt |
Closest papers by meaning
- Dynamic Mixture of Curriculum LoRA Experts for Continual Multimodal Instruction Tuning (ICML 2025 · similarity 0.87)
- Adapt-$\infty$: Scalable Continual Multimodal Instruction Tuning via Dynamic Data Selection (ICLR 2025 · similarity 0.86)
- Dynamic Cross-Modal Prompt Generation for Multimodal Continual Instruction Tuning (NeurIPS 2026 · similarity 0.85)
- Learning to Route: Per-Sample Adaptive Routing for Multimodal Multitask Prediction (NeurIPS 2025 · similarity 0.85)
- Improved Baselines with Visual Instruction Tuning (CVPR 2024 · similarity 0.85)
- FLAME: Adaptive Mixture-of-Experts for Continual Multimodal Multi-Task Learning (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
