AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

MultiMoDN—Multimodal, Multi-Task, Interpretable Modular Networks

Vinitra Swamy, Malika Satayeva, Jibril Frej, Thierry Bossy, Thijs Vogels, Martin Jaggi, Tanja Käser, Mary-Anne Hartley

NeurIPS 2023 · Vision-language and multimodal › Cross-modal learning · approach: Multimodal sentiment analysis · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Replace parallel concatenation with sequential modular fusionMultimodal prediction across MIMIC, EDU, and Weather tasks · 10 real-world tasksBAC and MSENo clear effect
Train task-specific decoders jointly rather than as separate single-task modelsMultiModN · MIMIC, EDU, and Weather taskstask performanceNo clear effect
Use sequential modular fusion instead of P-FusionMIMIC Cardiomegaly · MIMIC CardiomegalyAUROC: 20% improvementImproved
Use sequential modular fusion instead of P-Fusion under label-flipped MNARMIMIC task 1 · MIMIC, 80% MNAR label-flipped testAUROC: 0.385 AUROC for P-Fusion; MultiModN only loses 10% in MNAR flipMixed
Vary batch size (ablation)MultiModN · EDU student success predictionperformanceNo clear effect
Increase dropout on medium and large models (ablation)MultiModN · EDU student success predictionperformanceHurt
Use a smaller hidden layer in the large model (ablation)MultiModN · EDU student success predictionperformanceImproved
Use an excessively small state representation (sizes 1 or 5) (ablation)MultiModN · EDU student success predictionperformanceHurt
Use an excessively large state representation (size 100) (ablation)MultiModN · EDU student success predictionperformanceHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy