Research map / Vision-language and multimodal / Cross-modal learning
MMTM: Multimodal Transfer Module for CNN Fusion
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MMTM instead of late fusion | I3D RGB+Depth gesture recognition · EgoGesture | accuracy: 92.78 -> 93.51 | Helped |
| use MMTM instead of late fusion | I3D multimodal gesture recognition · NVGesture RGB+Depth | accuracy: 84.43 -> 86.31 | Helped |
| use MMTM instead of late fusion | I3D multimodal gesture recognition · NVGesture RGB+Opt. flow | accuracy: 84.43 -> 84.85 | Helped |
| use MMTM instead of late fusion | I3D multimodal gesture recognition · NVGesture RGB+Depth+Opt. flow | accuracy: 85.68 -> 86.93 | Helped |
| use MMTM instead of channel-wise concatenation | audio-visual speech enhancement · VoxCeleb2, 3 simultaneous speakers | PESQ: 2.67 -> 2.73; 0.06 PESQ | Helped |
| use MMTM instead of channel-wise concatenation | audio-visual speech enhancement · VoxCeleb2, 3 simultaneous speakers | STOI: 0.938 -> 0.941 | Helped |
| use MMTM instead of late fusion | I3D + HCN action recognition · NTU-RGBD cross-subject | accuracy: 91.56 -> 91.99 | Helped |
| use MMTM instead of MFAS | Inflated ResNet-50 + HCN action recognition · NTU-RGBD cross-subject | accuracy: 90.04 -> 90.11 | Helped |
| replace global squeeze with spatially retained convolutional fusion (ablation) | MMTM gesture recognition · NVGesture RGB+Depth | accuracy: 86.31 -> 84.43 | Hurt |
| replace global squeeze and gating with convolutional fusion and sum (ablation) | MMTM gesture recognition · NVGesture RGB+Depth | accuracy: 86.31 -> 84.65 | Hurt |
| replace cross-modal fusion with self-excitation in each stream (ablation) | MMTM gesture recognition · NVGesture RGB+Depth | accuracy: 86.31 -> 85.06 | Hurt |
| apply MMTM to a different number of inception modules, including lower-level features (ablation) | I3D RGB+Depth gesture recognition · NVGesture | accuracy | Mixed |
Closest papers by meaning
- Beyond Modality Fusion: Deep Ensembles for Multimodal Classification (NeurIPS 2026 · similarity 0.86)
- Fusion or Confusion? Multimodal Complexity Is Not All You Need (NeurIPS 2026 · similarity 0.85)
- AlignMamba: Enhancing Multimodal Mamba with Local and Global Cross-modal Alignment (CVPR 2025 · similarity 0.85)
- Modality-Aware SAM: Sharpness-Aware-Minimization Driven Gradient Modulation for Harmonized Multimodal Learning (NeurIPS 2025 · similarity 0.84)
- Deep Correlated Prompting for Visual Recognition with Missing Modalities (NeurIPS 2024 · similarity 0.84)
- Multi-Modal Object Re-identification via Sparse Mixture-of-Experts (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.