AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

Investigating Inner Properties of Multimodal Representation and Semantic Compositionality With Brain-Based Componential Semantics

Shaonan Wang, Jiajun Zhang, Nan Lin, Chengqing Zong

AAAI 2018 · Vision-language and multimodal › Human and embodied modeling · approach: Language-grounded visual concept learning · 10 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use Addition compositionText adjective-noun composition · TextQ1 rank: 5Improved
Use Matrix compositionVisual adjective-noun composition · ImageQ1 rank: 6Improved
Use Matrix compositionMultimodal adjective-noun composition · MultimodalQ1 rank: 0Improved
Replace Addition with Multiplication compositionText adjective-noun composition · TextQ1 rank: 5 -> 1332Hurt
Use multimodal rather than single-modality phrase representationsComposition models · Multimodalrank evaluationImproved
Add visual input to linguistic word representationsRidge multimodal representations · Brain-based sensory and motor propertiesRSA correlationImproved
Add visual input to linguistic word representationsRidge multimodal representations · Brain-based abstract propertiesRSA correlationHurt
Use MMskip multimodal representations instead of linguistic vectorsMMskip · Brain-based semantic propertiesRSA correlationNo clear effect
Use visual input for abstract conceptsMultimodal representations · Abstract wordsRSA correlationHurt
Use either linguistic or visual input for adjective-noun compositionSemantic composition · Adjective-noun phrasesProperty difference patternNo clear effect
Use parameterized composition models rather than Addition or MultiplicationMultimodal composition · 19 phrase categoriesMean property differenceImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy