Research map / Vision-language and multimodal / Human and embodied modeling
CSI-TextBench: A Dataset and Benchmark for Language-Grounded Ambient Sensing Perception
Guozhen Zhu, Yuqian Hu, Sakila S Jayaweera, Wei-Hsiang Wang, Beibei Wang, K. Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CSI-TextBench, a dataset and benchmark pairing WiFi channel state information with language for language-grounded indoor ambient sensing perception.
Problem
Ambient WiFi CSI captures indoor human activity and environmental context passively, but it remains weakly connected to language-aligned multimodal systems.
Approach
A benchmark that frames WiFi CSI perception as language-grounded tasks, pairing CSI signals with text so that language-aligned models can be evaluated on indoor sensing.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use contrastive CLIP InfoNCE instead of Barlow Twins for CSI-text alignment | CSI-text dual encoder · Fall detection zero-shot | accuracy (%): 59.8 -> 94.2 | Improved |
| use CLIP InfoNCE instead of SigLIP on 276-class SignFi | CSI-text dual encoder · SignFi zero-shot | accuracy (%): 71.8 -> 97.0 | Improved |
| contrastive CSI-text alignment versus supervised classifier with same encoder | CLIP InfoNCE · HAR zero-shot | accuracy (%): 97.9 -> 97.8 | No clear effect |
| swap ResNet-18 1D CSI encoder for ViT-1D or PatchTST (ablation) | CLIP InfoNCE on HAR · HAR zero-shot | accuracy (%): 98.7% -> 95.5% (ViT-1D), 94.1% (PatchTST) | Hurt |
| swap frozen CLIP text encoder for MiniLM-L6-v2 (ablation) | CLIP InfoNCE on HAR · HAR zero-shot | accuracy (%): 98.7% -> 90.2% | Hurt |
| add Level 2 compositional captions to Level 1 for single-task training (ablation) | CLIP InfoNCE · HAR and Proximity zero-shot | accuracy (%): difference at most 0.3 points | No clear effect |
| anchor expert-language prompts in a natural sentence frame (ablation) | template-trained CLIP InfoNCE · HAR open-vocabulary | accuracy (%): 59.1 -> 96.2 | Improved |
| anchor sensor-grounded prompts in a natural sentence frame (ablation) | template-trained CLIP InfoNCE · HAR open-vocabulary | accuracy (%): 35.4 -> 77.1 | Improved |
| train on human-written free-form text instead of templates (ablation) | CLIP InfoNCE on HAR · HAR zero-shot, specialized prompts | accuracy (%): 59.1 -> 87.0 (expert raw), 35.4 -> 80.2 (sensor raw) | Improved |
| train on human-written free-form text instead of templates (ablation) | CLIP InfoNCE on HAR · HAR zero-shot, ordinary prompts | accuracy (%): 95.9 -> 94.5 (train template), 96.0 -> 93.9 (layperson) | Hurt |
| joint multitask training on all 7 tasks with shared encoder | CLIP InfoNCE · Proximity zero-shot | accuracy (%): 94.6 -> 84.5 | Hurt |
| retrieve composite scene descriptions jointly instead of independent per-attribute QA | CLIP InfoNCE multitask model · Composite scenes (140) | joint/top-1 accuracy (%): 70.4 -> 86.9 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- LanguageBind — LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment (ICLR 2024) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Experimental setup
Benchmarks: CSI-TextBench
Closest papers by meaning
- CSI-Bench: A Large-Scale In-the-Wild Dataset for Multi-task WiFi Sensing (NeurIPS 2025 · similarity 0.87)
- SensorLM: Learning the Language of Wearable Sensors (NeurIPS 2025 · similarity 0.84)
- HoloLLM: Multisensory Foundation Model for Language-Grounded Human Sensing and Reasoning (NeurIPS 2025 · similarity 0.84)
- From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMs (CVPR 2026 · similarity 0.83)
- SING: Spatial Context in Large Language Model for Next-Gen Wearables (ICML 2025 · similarity 0.82)
- Limitations in Employing Natural Language Supervision for Sensor-Based Human Activity Recognition - And Ways to Overcome Them (AAAI 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
