Research map / Vision-language and multimodal / Cross-modal learning
M-SpecGene: Generalized Foundation Model for RGBT Multispectral Vision
Kailai Zhou, Fuqiang Yang, Shixian Wang, Bihan Wen, Chongde Zi, Linsen Chen, Qiu Shen, Xun Cao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace random masking with GMM-CMSS progressive masking (ablation) | M-SpecGene pre-training · FLIR | mAP50: 83.8 -> 84.8 | Improved |
| mask only in a single CMSS information-density region (low or high) (ablation) | M-SpecGene pre-training · FLIR | mAP50: 83.8 -> 83.6 (low), 83.4 (high) | Hurt |
| use Siamese architecture with shared encoder and two decoders instead of vanilla MAE (ablation) | M-SpecGene pre-training · FLIR | mAP50: 83.1 -> 83.8 | Improved |
| concatenate RGB and thermal along channels in pre-training (ablation) | M-SpecGene pre-training · FLIR | mAP50: 83.1 -> 80.1 | Hurt |
| M-SpecGene RGBT pre-training instead of ImageNet MAE (ablation) | ViT encoder · FLIR | mAP50: 82.8 -> 84.8 | Improved |
| M-SpecGene pre-training instead of from scratch for feature matching (ablation) | ViT-S LoFTR · LLVIP feature matching | AUC@10deg: 41.2 -> 48.2 | Improved |
| ImageNet supervised or MAE pre-training instead of scratch for feature matching (ablation) | ViT-S LoFTR · LLVIP feature matching | AUC@10deg: 41.2 -> 40.3 (Sup.), 37.0 (MAE) | Hurt |
| decoder depth 4 versus 2 or 8 (ablation) | M-SpecGene pre-training · FLIR | mAP50: 84.1 -> 84.8 (2 vs 4), 84.5 (8) | Improved |
| masking ratio 90% versus 85% or 95% (ablation) | M-SpecGene pre-training · FLIR | mAP50: 84.4 / 84.1 -> 84.8 | Improved |
| M-SpecGene ViT-B versus UniRGB-IR on KAIST | RGBT detection · KAIST (All) | MR^-2: 25.21 -> 23.74 | Improved |
| simple concatenation fusion with M-SpecGene on small FMB dataset | segmentation · FMB | mIoU (%): 61.2 (MRFS) vs 60.1 | No clear effect |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 0 of 1 on most shared results.
- MRFS — MRFS: Mutually Reinforcing Image Fusion and Segmentation (CVPR 2024) · better on 0 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Causal Mode Multiplexer: A Novel Framework for Unbiased Multispectral Pedestrian Detection (CVPR 2024 · similarity 0.86)
- RGB-to-Polarization Estimation: A New Task and Benchmark Study (NeurIPS 2025 · similarity 0.85)
- LESSViT: Robust Hyperspectral Representation Learning under Spectral Configuration Shift (NeurIPS 2026 · similarity 0.85)
- ThermalGen: Style-Disentangled Flow-Based Generative Models for RGB-to-Thermal Image Translation (NeurIPS 2025 · similarity 0.85)
- Spectral and Polarization Vision: Spectro-polarimetric Real-world Dataset (CVPR 2024 · similarity 0.84)
- S2MAE: A Spatial-Spectral Pretraining Foundation Model for Spectral Remote Sensing Data (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
