Research map / Vision-language and multimodal / Cross-modal learning
Channel Exchanging Networks for Multimodal and Multitask Dense Image Prediction
Yikai Wang, Fuchun Sun, Wenbing Huang, Fengxiang He, Dacheng Tao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| share convolutions across modalities while keeping BNs private (ablation) | CEN baseline · NYUDv2 | Ensemble Mean IoU: 47.6 -> 48.0 | Improved |
| share BN layers across modalities as well (ablation) | CEN baseline · NYUDv2 | Ensemble Mean IoU: 48.0 -> 45.2 | Hurt |
| add l1-guided directed channel exchanging (ablation) | shared convs, unshared BNs, half-channel l1 · NYUDv2 | Ensemble Mean IoU: 47.6 -> 51.1 | Improved |
| exchange a fixed 30% of channels without l1 (ablation) | shared convs, unshared BNs · NYUDv2 | Ensemble Mean IoU: 48.0 -> 47.2 | Hurt |
| exchange random channels like ShuffleNet without l1 (ablation) | shared convs, unshared BNs · NYUDv2 | Ensemble Mean IoU: 48.0 -> 46.8 | Hurt |
| restrict l1 and exchanging to half of the channels instead of all channels (ablation) | CEN · NYUDv2 | Ensemble Mean IoU: 49.8 -> 51.1 | Improved |
| share convolutions in CEN with half-channel l1 and exchange (ablation) | CEN · NYUDv2 | Ensemble Mean IoU: 48.5 -> 51.1 | Improved |
| replace attention/concat/alignment fusion with CEN | RefineNet ResNet101 · NYUDv2 | Ensemble Mean IoU: 49.1 -> 51.1 | Improved |
| CEN instead of self-attention in fusion parameters | RefineNet ResNet101 · NYUDv2 | Params used for fusion (M): 112.8 -> 0.0 | Improved |
| increase number of input modalities from one to four (ablation) | CEN · Taskonomy to RGB | FID: 84.33 -> 60.90 -> 57.19 | Improved |
| use cycle multimodal fusion with shared encoder instead of independent CEN (ablation) | CEN · Taskonomy Shade+Texture to RGB | FID: 62.63 -> 61.03 | Improved |
| randomly sample one flow per step instead of all flows (CEN-random) (ablation) | CEN-cycle · Taskonomy Shade+Texture to RGB | FID: 62.63 -> 73.27 | Hurt |
Closest papers by meaning
- Deep Multimodal Fusion by Channel Exchanging (NeurIPS 2020 · similarity 0.93)
- MultiMoDN—Multimodal, Multi-Task, Interpretable Modular Networks (NeurIPS 2023 · similarity 0.87)
- VMT-Adapter: Parameter-Efficient Transfer Learning for Multi-Task Dense Scene Understanding (AAAI 2024 · similarity 0.86)
- Fusion or Confusion? Multimodal Complexity Is Not All You Need (NeurIPS 2026 · similarity 0.85)
- Keep the Balance: A Parameter-Efficient Symmetrical Framework for RGB+X Semantic Segmentation (CVPR 2025 · similarity 0.85)
- CATP: Contextually Adaptive Token Pruning for Efficient and Enhanced Multimodal In-Context Learning (AAAI 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
