AI Paper Atlas

Research map / Multi-agent systems and games / Human-AI and goal-conditioned

V-IRL: Grounding Virtual Intelligence in Real Life

Jihan Yang, Runyu Ding, Ellis L Brown, Xiaojuan Qi, Saining Xie

ECCV 2024 · Multi-agent systems and games › Human-AI and goal-conditioned · approach: Human-AI coordination agents · 31 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add active detection (adjust ego-pose and focal length) to the urban robot (ablation)RX-399 open-world detection · Hong Kong street viewsaccuracy: 0.10 -> 0.63Improved
add active detection (adjust ego-pose and focal length) to the urban robot (ablation)RX-399 open-world detection · New York City street viewsaccuracy: 0.30 -> 0.71Improved
reclassify GLIP proposals with CLIP instead of using end-to-end open-world detectorsPlace detection · V-IRL Place DetectionAR20: 9.0 -> 23.7Improved
scale CLIP image resolution and model size (ablation)CLIP place recognition · V-IRL Place RecognitionmAcc (%): 18.2 -> 41.3Improved
replace oracle landmark information with vision-model recognition (ablation)VLN agent with GPT-4 · V-IRL VLN-minisuccess rate: 1.0 -> 0.44Hurt
use larger CLIP variant for landmark recognition (B/32 to L/14@336px) (ablation)VLN agent · V-IRL VLN-minisuccess rate: 0.22 -> 0.44Improved
use LLaVA-1.5-13B instead of CLIP for landmark recognition (ablation)VLN agent · V-IRL VLN-minisuccess rate: 0.44 -> 0.11Hurt
give the concierge agent real travel times from the platform instead of an ungrounded LLM (ablation)Diego concierge · NYC itinerary (qualitative)itinerary feasibility: 30 minutes -> 60--100 minutesImproved
scale up LLaVA generation: LLaVA-NeXT versus LLaVA-1.5 and LLaVAMLLMs · V-IRL Place VQAmAcc (%): 61.9 -> 65.9Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy