Research map / Vision-language and multimodal / Cross-modal learning
LEO-Fuse: A Modality- and Task-Agnostic Universal Framework for Multimodal Human Sensing
Emrecan Aslan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add local routing head (ablation) | LEO-Fuse · MM-Fi (31-subset average) | MPJPE (mm): 88.31 -> 75.53 | Improved |
| add unimodal deep supervision (ablation) | LEO-Fuse · MM-Fi (31-subset average) | MPJPE (mm): 83.44 -> 75.53 | Improved |
| use K-based weighted subset sampling instead of uniform (ablation) | LEO-Fuse · MM-Fi (31-subset average) | MPJPE (mm): 79.54 -> 75.53 | Improved |
| add mode-aware loss balancing (ablation) | LEO-Fuse · MM-Fi (31-subset average) | MPJPE (mm): 78.75 -> 75.53 | Improved |
| add gate supervision (ablation) | LEO-Fuse · MM-Fi (31-subset average) | MPJPE (mm): 76.86 -> 75.53 | Improved |
| shrink pose head from 14.46M to 4.49M parameters (ablation) | LEO-Fuse · MM-Fi (31-subset average) | MPJPE (mm): 75.53 -> 76.02 | No clear effect |
| train one universal model across subsets vs dedicated LiDAR model | LEO-Fuse · MM-Fi LiDAR singleton | MPJPE (mm): 121.02 -> 77.93 | Improved |
| train one universal model across subsets vs dedicated mmWave model | LEO-Fuse · MM-Fi mmWave singleton | MPJPE (mm): 135.51 -> 119.97 | Improved |
| train one universal model vs dedicated WiFi model | LEO-Fuse · MM-Fi WiFi singleton | MPJPE (mm): 199.09 -> 203.38 | Hurt |
| train frozen encoders with HAR labels instead of reusing HPE encoders | LEO-Fuse · MM-Fi HAR (31-subset average) | top-1 accuracy (%): 93.4 -> 95.7 | Improved |
| add more modalities (K=1 to K=4) | LEO-Fuse · MM-Fi HPE | MPJPE (mm): 106.21 -> 61.08 | Improved |
| add WiFi to D+L+R+I (full five modalities) | LEO-Fuse · MM-Fi HPE | MPJPE (mm): 57.22 -> 58.82 | Hurt |
Closest papers by meaning
- X-Fi: A Modality-Invariant Foundation Model for Multimodal Human Sensing (ICLR 2025 · similarity 0.91)
- HoloLLM: Multisensory Foundation Model for Language-Grounded Human Sensing and Reasoning (NeurIPS 2025 · similarity 0.85)
- VCR: Learning Valid Contextual Representation for Incomplete Wearable Signals (NeurIPS 2026 · similarity 0.85)
- FreeMan: Towards Benchmarking 3D Human Pose Estimation under Real-World Conditions (CVPR 2024 · similarity 0.84)
- Ego4o: Egocentric Human Motion Capture and Understanding from Multi-Modal Input (CVPR 2025 · similarity 0.83)
- SensorLM: Learning the Language of Wearable Sensors (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
