AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

Learning to Compose Visual Relations

Nan Liu, Shuang Li, Yilun Du, Joshua B. Tenenbaum, Antonio Torralba

NeurIPS 2021 · Vision-language and multimodal › Human and embodied modeling · approach: Scene graph generation · 11 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
factorize each scene into separate relation-conditioned EBMs and compose their energiesrelational image generation · CLEVR 2R and 3Raccuracy (%): 2.46 -> 69.55; 0.54 -> 37.60Improved
factorize each scene into separate relation-conditioned EBMs and compose their energiesrelational image generation · iGibson 2R and 3Raccuracy (%): 2.24 -> 45.03; 0.60 -> 19.39Improved
use learned attribute embeddings instead of CLIP object embeddings (ablation)Ours · CLEVR image generation, 1R/2R/3Raccuracy (%): 94.79 -> 97.79; 48.42 -> 69.55; 18.00 -> 37.60Improved
use learned attribute embeddings instead of CLIP object embeddings (ablation)Ours · iGibson image generation, 1R/2Raccuracy (%): 74.02 -> 78.27; 43.03 -> 45.03Improved
use learned attribute embeddings instead of CLIP object embeddings (ablation)Ours · iGibson 3R image generationaccuracy (%): 19.59 -> 19.39Hurt
use learned attribute embeddings instead of CLIP object embeddings (ablation)Ours · CLEVR image editing, 1R/2R/3Raccuracy (%): 95.56 -> 97.52; 52.78 -> 65.88; 16.32 -> 32.38Improved
use learned attribute embeddings instead of CLIP object embeddings (ablation)Ours · iGibson image editing, 1R/2R/3Raccuracy (%): 78.12 -> 84.16; 32.84 -> 44.10; 12.66 -> 20.76Improved
encode the full relational sentence holistically rather than separately encoding relation triplet elements (ablation)conditional EBM · CLEVR image generation, 1R/2R/3Raccuracy (%): 94.79 -> 4.75; 48.42 -> 0.24; 18.00 -> 0.00Hurt
train StyleGAN2 (CLIP) on 1–3 relations rather than a single relation (ablation)StyleGAN2 (CLIP) · CLEVR image generation, 1R/2R/3Raccuracy (%): 65.98 -> 66.62; 9.56 -> 9.60; 1.78 -> 1.68Mixed
train StyleGAN2 (CLIP) on 1–3 relations rather than a single relation (ablation)StyleGAN2 (CLIP) · iGibson image generation, 1R/2R/3Raccuracy (%): 49.20 -> 36.94; 17.06 -> 13.42; 5.10 -> 6.86Mixed
use a holistic CLIP sentence encoder instead of decomposed relation descriptions (ablation)relational understanding · CLEVR image generationaccuracy (%): Ours (CLIP): 94.79/48.42/18.00; Full Sentence: 4.75/0.24/0.00Hurt
compose relations with the proposed method rather than StyleGAN2 (CLIP)relational image editing · CLEVR and iGibson 1R/2R/3Raccuracy (%)Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy