AI Paper Atlas

Research map / Vision-language and multimodal / Visual question answering

Perceptual Score: What Data Modalities Does Your Model Perceive?

Itai Gat, Idan Schwartz, Alex Schwing

NeurIPS 2021 · Vision-language and multimodal › Visual question answering · approach: Visual reasoning diagnostics · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train LMH on CSS-generated counterfactual samples with unmodified inputs (ablation)LMH · VQA-CPaccuracy: 57.54% (vs CSS 57.89, LMH 52.23)Improved
CSS counterfactual data shifts yes/no prior toward test distributionCSS · VQA-CP Yes/Noimage perceptual score P_V/Z_D: 0.71 vs LMH 20.58Hurt
use answer-only features instead of all SocialIQ modalities (ablation)SocialIQ baseline · SocialIQ valaccuracy: 64.84 -> 68.65Improved
change baseline to FGA with GloVe featuresSocialIQ baseline · SocialIQ valaccuracy: 64.84 -> 67.38Improved
untrained NLTK sentiment classifier on answer textNLTK-Sentiment · SocialIQ valaccuracy: 66.7% vs baseline 64.84Improved
lower image quality in audio-visual crowd countingaudio-visual model · DISCOaudio perceptual score P_A/Z_D: 9.22 -> 23.91Improved
compare ranking metric NDCG vs MRR for question perceptionLS vs FGA · VisDial v1.0question perceptual score P_Q/Z_{D,f}: MRR: FGA 51.26 vs LS 30.49; NDCG: LS 63.14 vs FGA 45.11Mixed
fine-tune LS on dense annotations (LS (CE)) vs LSLS · VisDial v1.0 MRRMRR: 69.00 -> 52.21Hurt
use caption modality in dialogLS, FGA, LS (CE) · VisDial v1.0caption perceptual score: 0.00 to 1.83No clear effect
model-normalize rather than task-normalize perceptual score for LMH (lowest accuracy)LMH · VQAv2image perceptual score: 48.64 vs LXMERT 47.16No clear effect
use nonlinear network instead of logistic regression on synthetic dataperceptual score · SyntheticP_a, P_b (avg): 0.39 -> 35.72 (P_a), 0.56 -> 36.8 (P_b)Improved
increase variance of modality c from 0 to 1perceptual score · SyntheticP_c: 0 -> 32.25Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy