Research map / Vision-language and multimodal / Human and embodied modeling
Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors
Yansen Wang, Ying Shen, Zhun Liu, Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Multimodal Shifting (ablation) | RAVEN · CMU-MOSI | MAE: 0.954 -> 0.915 | Improved |
| Add Multimodal Shifting (ablation) | RAVEN · CMU-MOSI | Corr: 0.666 -> 0.691 | Improved |
| Add Multimodal Shifting (ablation) | RAVEN · CMU-MOSI | Acc-2: 77.7 -> 78.0 | Improved |
| Model visual and acoustic subword sequences instead of averaging them (ablation) | RAVEN · CMU-MOSI | MAE: 0.934 -> 0.915 | Improved |
| Model visual and acoustic subword sequences instead of averaging them (ablation) | RAVEN · CMU-MOSI | Corr: 0.652 -> 0.691 | Improved |
| Model visual and acoustic subword sequences instead of averaging them (ablation) | RAVEN · CMU-MOSI | Acc-2: 73.9 -> 78.0 | Improved |
| Add both subword modeling and multimodal shifting (ablation) | RAVEN early-fusion variant · CMU-MOSI | MAE: 1.423 -> 0.915 | Improved |
| Add both subword modeling and multimodal shifting (ablation) | RAVEN early-fusion variant · CMU-MOSI | Corr: 0.116 -> 0.691 | Improved |
| Add both subword modeling and multimodal shifting (ablation) | RAVEN early-fusion variant · CMU-MOSI | Acc-2: 50.6 -> 78.0 | Improved |
| Use RAVEN instead of LMF | multimodal emotion recognition · IEMOCAP Happy | Acc-2: 87.3 vs 87.3? | Hurt |
Closest papers by meaning
- Debiased Multimodal Understanding for Human Language Sequences (AAAI 2025 · similarity 0.87)
- Less is More: Local Intrinsic Dimensions of Contextual Language Models (NeurIPS 2025 · similarity 0.85)
- AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Models (ICML 2025 · similarity 0.85)
- DLF: Disentangled-Language-Focused Multimodal Sentiment Analysis (AAAI 2025 · similarity 0.84)
- Text-Guided Nonverbal Enhancement Based on Modality-Invariant and -Specific Representations for Video Speaking Style Recognition (AAAI 2025 · similarity 0.84)
- EmergentTTS-Eval: Evaluating TTS Models on Complex Prosodic, Expressiveness, and Linguistic Challenges Using Model-as-a-Judge (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
