AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Visual Reasoning: From State to Transformation

Xin Hong, Yanyan Lan, Liang Pang, Jiafeng Guo, Xueqi Cheng

TPAMI 2023 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 9 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add REINFORCE with correctness reward (ablation)ResNet- -T · TRANCE EventAcc: 0.6553 -> 0.6729Improved
add REINFORCE with distance reward (ablation)ResNet- -T · TRANCE EventAcc: 0.6553 -> 0.6734Improved
add REINFORCE with correctness and distance rewards (ablation)ResNet- -T · TRANCE EventAcc: 0.6553 -> 0.6740Improved
use transformer decoder instead of GRUResNet- encoder · TRANCE EventAcc: 0.6057 -> 0.6553Improved
use early-fusion ResNet encoder instead of late-fusion BCNNGRU decoder · TRANCE BasicAcc: 0.9524 -> 0.9814Improved
use larger ResNet encoder instead of 4-layer CNNGRU decoder, subtraction fusion · TRANCE EventAcc: 0.4419 -> 0.6057Improved
vary final-state camera view (Event to View)ResNet- -T · TRANCEAcc: 0.6553 -> 0.6012Hurt
vary final-state camera view (Event to View)CNN- -G · TRANCEAcc: 0.4419 -> 0.2300Hurt
increase transformation length from 1 to 4 stepsCNN- -G · TRANCE EventAcc: 92%, 55%, 23%, and 8%Hurt
add pretrained CLIP weights (frozen) (ablation)ViT-B/32-T · TRANCOEMR: 0.0378 -> 0.0881Improved
add pretrained CLIP weights (frozen) (ablation)ViT-B/32-T · TRANCO tiny candidatesEMR@100: 0.1210 -> 0.1986Improved
further finetune pretrained CLIP weights (ablation)ViT-B/32-T · TRANCOEMR: 0.0881 -> 0.0748Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy