Research map / Vision-language and multimodal / Human and embodied modeling
Visual Interestingness Decoded: How GPT-4o Mirrors Human Interests
Fitim Abdullahu, Helmut Grabner
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| restrict to images where humans reach consensus | GPT-4o pairwise labels · Pairwise interestingness | agreement (%): 66.2% -> 73.8% | Improved |
| evaluate on dissent set instead of full set | GPT-4o pairwise labels · Pairwise interestingness | agreement (%): 66.2% -> 56.5% | Hurt |
| swap image order and keep only order-consistent GPT-4o answers | GPT-4o pairwise annotation · 2,500 Flickr image pairs | fraction consistent: 36% always picked second image; 64% (1,599 of 2,500) consistent | Hurt |
| add demographic system prompt (gender, continent, age) | GPT-4o · 116 image pairs | preference | No clear effect |
| ask relative pairwise question instead of single-image question | GPT-4o · Flickr images | label informativeness | Improved |
| use GPT-4o labels instead of aesthetic model labels | pairwise labelling · Human labels agreement | agreement (%): 68.3 -> 73.8 | Improved |
| use GPT-4o labels instead of CI model labels | pairwise labelling · Human labels agreement | agreement (%): 69.6 -> 73.8 | Improved |
| use GPT-4o labels instead of memorability labels | pairwise labelling · Human labels agreement | agreement (%): 35.5 -> 73.8 | Improved |
| use GPT-4o labels instead of CuPL zero-shot CLIP | pairwise labelling · Human labels agreement | agreement (%): 60.3 -> 73.8 | Improved |
| use GPT-4o labels instead of favorites count | pairwise labelling · Human labels agreement | agreement (%): 66.4 -> 73.8 | Improved |
| train ranker on GPT-4o labels instead of human labels, tested on human labels | CLIP learning-to-rank · Human test labels | accuracy (%): 77.5 -> 73.4 | Mixed |
| use DINOv2 embeddings instead of CLIP | learning-to-rank · Human and GPT-4o labels | accuracy (%): 67.1% for Humans and 68.3% for GPT-4o | Hurt |
Closest papers by meaning
- UniAR: A Unified model for predicting human Attention and Responses on visual content (NeurIPS 2024 · similarity 0.84)
- See What You Are Told: Visual Attention Sink in Large Multimodal Models (ICLR 2025 · similarity 0.83)
- ConTextual: Evaluating Context-Sensitive Text-Rich Visual Reasoning in Large Multimodal Models (ICML 2024 · similarity 0.83)
- PhD: A ChatGPT-Prompted Visual Hallucination Evaluation Dataset (CVPR 2025 · similarity 0.83)
- Cross-modal Associations in Vision and Language Models: Revisiting the Bouba-Kiki Effect (NeurIPS 2025 · similarity 0.83)
- Evaluating Multiview Object Consistency in Humans and Image Models (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
