AI Paper Atlas

Research map / Detection and segmentation / CNN features

Scene-Graph ViT: End-to-End Open-Vocabulary Visual Relationship Detection

Tim Salzmann, Markus Ryll, Alex Bewley, Matthias Minderer

ECCV 2024 · Detection and segmentation › CNN features · approach: Efficient DETR query design · 8 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace decoder-based relationship module (UniVRD) with encoder-only Relationship Attention, same data mixture (ablation)SG-ViT B/32 · VG150mR@100: 12.1 -> 17.4Improved
add simple predicate rebalancing fine-tuning (ablation)SG-ViT L/14 · VG150mR@100: 21.8 -> 26.1Improved
remove Objects365 detection data from training mixture (ablation)SG-ViT B/32 · VG150mR@100: 18.1 -> 15.3Hurt
remove Visual Genome from training mixture (ablation)SG-ViT B/32 · VG150mR@100: 18.1 -> 14.2Hurt
scale backbone from ViT-B/32 to ViT-L/14 (ablation)SG-ViT · VG150mR@100: 18.1 -> 21.8Improved
reduce top-k relationships at inference from 2^14 to 2^11 (ablation)SG-ViT · VRD speed and accuracyaccuracy and speed: speed 71-81% -> 90% of object detectorNo clear effect
share one MLP for subject and object embeddings (ablation)Relationship Attention · VRD trainingfinal scoreHurt
use SG-ViT instead of UniVRD on HICOHICO-DET · HICO test (L/14)mAP: 37.41 -> 38.11No clear effect
use SG-ViT instead of UniVRD on rare HOI classesHICO-DET · HICO test (L/14)mAPr: 28.90 -> 33.71Improved
add relationship attention to the object detector on pure detection dataOWL-ViT · LVISmAP (%): 22.1 -> 21.9No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy