AI Paper Atlas

Research map / Graph learning / Graph self-supervised learning

GRAB: A Challenging GRaph Analysis Benchmark for Large Multimodal Models

Jonathan Roberts, Kai Han, Samuel Albanie

ICCV 2025 · Graph learning › Graph self-supervised learning · approach: LLMs and graph foundation models · 5 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use multiple-choice format instead of single answer (ablation)Claude 3.5 Sonnet · GRAB Propertiesaccuracy (%): 41.8 -> 31.8Hurt
use multiple-choice format instead of single answer (ablation)Claude 3 Haiku · GRAB Propertiesaccuracy (%): 14.2 -> 23.6Improved
use multiple-choice format instead of single answer (ablation)LLaVA-1.5 13b · GRAB Propertiesaccuracy (%): 5.0 -> 22.4Improved
use multiple-choice format instead of single answer (ablation)Gemini 1.0 Pro Vision · GRAB Propertiesaccuracy (%): 20.2 -> 12.7Hurt
use Seaborn instead of Matplotlib plots (ablation)Claude 3.5 Sonnet · GRAB 100-question subsetaccuracy (%): 30.0 -> 26.0No clear effect
replace exact matching with LLM answer extraction (ablation)LMM evaluation · GRABaccuracy (%)No clear effect
increase task complexity from 0 to 3 functions/series/transforms (ablation)Claude 3.5 Sonnet and Gemini 1.5 Pro · GRAB Functions/Series/Transformsaccuracy (%)Hurt
require decimal precision instead of integer precision (ablation)LMMs · GRABaccuracy (%)Hurt
newer model release (Gemini 1.5 Flash to Gemini 2.0 Flash)Gemini Flash · GRAB Real taskaccuracy (%): 17.1 -> 19.6Improved
newer model release (Gemini 1.5 Flash to Gemini 2.5 Flash)Gemini Flash · GRAB Real taskaccuracy (%): 17.1 -> 29.4Improved
newer frontier reasoning models vs older baselinesGPT-5 vs GPT-4o · GRAB-Liteaccuracy (%): 12.6 -> 46.2Improved
newer models on Transforms taskGPT-5 · GRAB-Lite Transformsaccuracy (%): 6.0 -> 63.0Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy