Research map / Vision-language and multimodal / Human and embodied modeling
Learning to Compose Visual Relations
Nan Liu, Shuang Li, Yilun Du, Joshua B. Tenenbaum, Antonio Torralba
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| factorize each scene into separate relation-conditioned EBMs and compose their energies | relational image generation · CLEVR 2R and 3R | accuracy (%): 2.46 -> 69.55; 0.54 -> 37.60 | Improved |
| factorize each scene into separate relation-conditioned EBMs and compose their energies | relational image generation · iGibson 2R and 3R | accuracy (%): 2.24 -> 45.03; 0.60 -> 19.39 | Improved |
| use learned attribute embeddings instead of CLIP object embeddings (ablation) | Ours · CLEVR image generation, 1R/2R/3R | accuracy (%): 94.79 -> 97.79; 48.42 -> 69.55; 18.00 -> 37.60 | Improved |
| use learned attribute embeddings instead of CLIP object embeddings (ablation) | Ours · iGibson image generation, 1R/2R | accuracy (%): 74.02 -> 78.27; 43.03 -> 45.03 | Improved |
| use learned attribute embeddings instead of CLIP object embeddings (ablation) | Ours · iGibson 3R image generation | accuracy (%): 19.59 -> 19.39 | Hurt |
| use learned attribute embeddings instead of CLIP object embeddings (ablation) | Ours · CLEVR image editing, 1R/2R/3R | accuracy (%): 95.56 -> 97.52; 52.78 -> 65.88; 16.32 -> 32.38 | Improved |
| use learned attribute embeddings instead of CLIP object embeddings (ablation) | Ours · iGibson image editing, 1R/2R/3R | accuracy (%): 78.12 -> 84.16; 32.84 -> 44.10; 12.66 -> 20.76 | Improved |
| encode the full relational sentence holistically rather than separately encoding relation triplet elements (ablation) | conditional EBM · CLEVR image generation, 1R/2R/3R | accuracy (%): 94.79 -> 4.75; 48.42 -> 0.24; 18.00 -> 0.00 | Hurt |
| train StyleGAN2 (CLIP) on 1–3 relations rather than a single relation (ablation) | StyleGAN2 (CLIP) · CLEVR image generation, 1R/2R/3R | accuracy (%): 65.98 -> 66.62; 9.56 -> 9.60; 1.78 -> 1.68 | Mixed |
| train StyleGAN2 (CLIP) on 1–3 relations rather than a single relation (ablation) | StyleGAN2 (CLIP) · iGibson image generation, 1R/2R/3R | accuracy (%): 49.20 -> 36.94; 17.06 -> 13.42; 5.10 -> 6.86 | Mixed |
| use a holistic CLIP sentence encoder instead of decomposed relation descriptions (ablation) | relational understanding · CLEVR image generation | accuracy (%): Ours (CLIP): 94.79/48.42/18.00; Full Sentence: 4.75/0.24/0.00 | Hurt |
| compose relations with the proposed method rather than StyleGAN2 (CLIP) | relational image editing · CLEVR and iGibson 1R/2R/3R | accuracy (%) | Improved |
Closest papers by meaning
- Weakly-Supervised Learning of Visual Relations (ICCV 2017 · similarity 0.89)
- Relation Rectification in Diffusion Model (CVPR 2024 · similarity 0.89)
- LinkNet: Relational Embedding for Scene Graph (NeurIPS 2018 · similarity 0.88)
- Joint Modeling of Visual Objects and Relations for Scene Graph Generation (NeurIPS 2021 · similarity 0.88)
- ART: Adaptive Relation Tuning for Generalized Relation Prediction (ICCV 2025 · similarity 0.88)
- Vision Relation Transformer for Unbiased Scene Graph Generation (ICCV 2023 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
