Research map / Vision-language and multimodal / Cross-modal learning
From 2D Grids to 1D Tokens: Reforming Shared Representations for Multimodal Image Fusion
Yuchen Xian, Yunqiu Xu, Yang He, Yi Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace 2D shared grid with 1D token shared representation | CDDFuse-style fusion · M3FD | SSIM: 1.02 -> 1.49 | Improved |
| add sparse edit at position 12 only (ablation) | 1D token fusion · Sparse token manipulation | EI: 34.39 -> 36.01 | Improved |
| add sparse edit at position 18 only (ablation) | 1D token fusion · Sparse token manipulation | EI: 34.39 -> 36.87 | Improved |
| edit positions 12 and 18 jointly (ablation) | 1D token fusion · Sparse token manipulation | EI: 34.39 -> 37.42 | Improved |
| use 32 tokens instead of 128 (ablation) | TiTok fusion · Token length ablation | EI: 32.13 -> 37.42 | Improved |
| use 32 tokens instead of 64 (ablation) | TiTok fusion · Token length ablation | EI: 34.57 -> 37.42 | Improved |
| use TiTok instead of DINOv3 as interface (ablation) | 1D token fusion · M3FD | SD: 24.35 -> 44.54 | Improved |
| use TiTok instead of CLIP as interface (ablation) | 1D token fusion · M3FD | SD: 27.29 -> 44.54 | Improved |
| use TiTok instead of CLIP as interface (ablation) | 1D token fusion · MIF | EN: 6.19 -> 4.27 | Hurt |
| use two editing slots instead of one (ablation) | STE selector · M3FD | SD: 44.60 -> 47.35 | Improved |
| use three slots instead of two (ablation) | STE selector · M3FD | SD: 47.35 -> 47.02 | No clear effect |
| use four slots instead of two (ablation) | STE selector · M3FD | SD: 47.35 -> 46.55 | No clear effect |
Closest papers by meaning
- TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generation (CVPR 2025 · similarity 0.87)
- PAIR Diffusion: A Comprehensive Multimodal Object-Level Image Editor (CVPR 2024 · similarity 0.85)
- G$^2$TR: Generation-Guided Visual Token Reduction for Separate-Encoder Unified Multimodal Models (NeurIPS 2026 · similarity 0.85)
- Which Tokens to Merge? Diffusion Dynamics for Efficient Image Generation (NeurIPS 2026 · similarity 0.85)
- When Integral Meets Decomposition: A Signal-Level Self-Supervised Feature Decompose Paradigm for Multi-Modal Image Fusion (NeurIPS 2026 · similarity 0.85)
- From Pixels to Tokens: Byte-Pair Encoding on Quantized Visual Modalities (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
