AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

Channel Exchanging Networks for Multimodal and Multitask Dense Image Prediction

Yikai Wang, Fuchun Sun, Wenbing Huang, Fengxiang He, Dacheng Tao

TPAMI 2022 · Vision-language and multimodal › Cross-modal learning · approach: Multimodal fusion with missing modalities · 54 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
share convolutions across modalities while keeping BNs private (ablation)CEN baseline · NYUDv2Ensemble Mean IoU: 47.6 -> 48.0Improved
share BN layers across modalities as well (ablation)CEN baseline · NYUDv2Ensemble Mean IoU: 48.0 -> 45.2Hurt
add l1-guided directed channel exchanging (ablation)shared convs, unshared BNs, half-channel l1 · NYUDv2Ensemble Mean IoU: 47.6 -> 51.1Improved
exchange a fixed 30% of channels without l1 (ablation)shared convs, unshared BNs · NYUDv2Ensemble Mean IoU: 48.0 -> 47.2Hurt
exchange random channels like ShuffleNet without l1 (ablation)shared convs, unshared BNs · NYUDv2Ensemble Mean IoU: 48.0 -> 46.8Hurt
restrict l1 and exchanging to half of the channels instead of all channels (ablation)CEN · NYUDv2Ensemble Mean IoU: 49.8 -> 51.1Improved
share convolutions in CEN with half-channel l1 and exchange (ablation)CEN · NYUDv2Ensemble Mean IoU: 48.5 -> 51.1Improved
replace attention/concat/alignment fusion with CENRefineNet ResNet101 · NYUDv2Ensemble Mean IoU: 49.1 -> 51.1Improved
CEN instead of self-attention in fusion parametersRefineNet ResNet101 · NYUDv2Params used for fusion (M): 112.8 -> 0.0Improved
increase number of input modalities from one to four (ablation)CEN · Taskonomy to RGBFID: 84.33 -> 60.90 -> 57.19Improved
use cycle multimodal fusion with shared encoder instead of independent CEN (ablation)CEN · Taskonomy Shade+Texture to RGBFID: 62.63 -> 61.03Improved
randomly sample one flow per step instead of all flows (CEN-random) (ablation)CEN-cycle · Taskonomy Shade+Texture to RGBFID: 62.63 -> 73.27Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy