Research map / Vision-language and multimodal / Cross-modal learning
MultiMoDN—Multimodal, Multi-Task, Interpretable Modular Networks
Vinitra Swamy, Malika Satayeva, Jibril Frej, Thierry Bossy, Thijs Vogels, Martin Jaggi, Tanja Käser, Mary-Anne Hartley
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace parallel concatenation with sequential modular fusion | Multimodal prediction across MIMIC, EDU, and Weather tasks · 10 real-world tasks | BAC and MSE | No clear effect |
| Train task-specific decoders jointly rather than as separate single-task models | MultiModN · MIMIC, EDU, and Weather tasks | task performance | No clear effect |
| Use sequential modular fusion instead of P-Fusion | MIMIC Cardiomegaly · MIMIC Cardiomegaly | AUROC: 20% improvement | Improved |
| Use sequential modular fusion instead of P-Fusion under label-flipped MNAR | MIMIC task 1 · MIMIC, 80% MNAR label-flipped test | AUROC: 0.385 AUROC for P-Fusion; MultiModN only loses 10% in MNAR flip | Mixed |
| Vary batch size (ablation) | MultiModN · EDU student success prediction | performance | No clear effect |
| Increase dropout on medium and large models (ablation) | MultiModN · EDU student success prediction | performance | Hurt |
| Use a smaller hidden layer in the large model (ablation) | MultiModN · EDU student success prediction | performance | Improved |
| Use an excessively small state representation (sizes 1 or 5) (ablation) | MultiModN · EDU student success prediction | performance | Hurt |
| Use an excessively large state representation (size 100) (ablation) | MultiModN · EDU student success prediction | performance | Hurt |
Closest papers by meaning
- Fusion or Confusion? Multimodal Complexity Is Not All You Need (NeurIPS 2026 · similarity 0.89)
- CyIN: Cyclic Informative Latent Space for Bridging Complete and Incomplete Multimodal Learning (NeurIPS 2025 · similarity 0.88)
- Revealing Vision-Language Integration in the Brain with Multimodal Networks (ICML 2024 · similarity 0.88)
- MSP: Modality Self-Play for Training Multimodal LLMs Without Paired Cross-Modal Data (NeurIPS 2026 · similarity 0.88)
- When to Align, When to Predict: A Phase Diagram for Multimodal Learning (NeurIPS 2026 · similarity 0.88)
- Beyond Modality Fusion: Deep Ensembles for Multimodal Classification (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
