Research map / Vision-language and multimodal / Cross-modal learning
Learning Modality-agnostic Representation for Semantic Segmentation from Any Modalities
Xu Zheng, Yuanhuiyi Lyu, Lin Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add semantic KD loss L_se (RGB only) (ablation) | Any2Seg (SegFormer-B0) · DELIVER RGB | mIoU: 51.39 -> 53.03 | Improved |
| add L_se with R+E input (ablation) | Any2Seg (SegFormer-B0) · DELIVER R+E | mAcc: mAcc 60.36 -> 60.20; mIoU 52.21 -> 53.11 | Mixed |
| add inter-modal correlation KD L_cr (R+D) (ablation) | Any2Seg (SegFormer-B0) · DELIVER R+D | mIoU: +0.83 mIoU, +2.46 Acc | Improved |
| add all three losses vs base loss only (ablation) | Any2Seg (SegFormer-B0) · DELIVER R+D (MSS) | mIoU: 61.93 -> 63.74 | Improved |
| add fine-grained feature selection to MFF (ablation) | MFF (SegFormer-B0) · DELIVER 4 modalities (MISS) | mIoU: 35.47 -> 44.49 | Improved |
| add fine-grained feature selection to MFF (ablation) | MFF (SegFormer-B0) · MCubeS 4 modalities (MSS) | mIoU: 39.63 -> 42.49 | Improved |
| add residual connection without selection (ablation) | MFF (SegFormer-B0) · MCubeS 4 modalities (MISS) | mIoU: 27.88 -> 25.10 | Hurt |
| add residual connection to selection (ablation) | MFF (SegFormer-B0) · DELIVER 4 modalities (MISS) | mIoU: 44.49 -> 47.79 | Improved |
| replace CMNeXt fusion with Any2Seg (LSCD+MFF) | DELIVER system-level MISS · DELIVER MISS mean | mIoU: 25.25 -> 45.04 | Improved |
| Any2Seg vs CMNeXt on R+L | Any2Seg · DELIVER MSS R+L | mIoU: 58.04 -> 57.92 | Hurt |
| Any2Seg in sparse modality (Event, LiDAR only) inputs | Any2Seg · DELIVER system-level MISS | mIoU: L: 0.72 -> 0.31 | Mixed |
Closest papers by meaning
- Keep the Balance: A Parameter-Efficient Symmetrical Framework for RGB+X Semantic Segmentation (CVPR 2025 · similarity 0.91)
- OmniSegmentor: A Flexible Multi-Modal Learning Framework for Semantic Segmentation (NeurIPS 2025 · similarity 0.91)
- OGP-Net: Optical Guidance Meets Pixel-Level Contrastive Distillation for Robust Multi-Modal and Missing Modality Segmentation (AAAI 2025 · similarity 0.89)
- UniMRSeg: Unified Modality-Relax Segmentation via Hierarchical Self-Supervised Compensation (NeurIPS 2025 · similarity 0.88)
- Hierarchical Intra-modal Correlation Learning for Label-free 3D Semantic Segmentation (CVPR 2024 · similarity 0.87)
- MMRL: Multi-Modal Representation Learning for Vision-Language Models (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
