AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

StyleBabel: Artistic Style Tagging and Captioning

Dan Ruta, Andrew Gilbert, Pranav Aggarwal, Naveen Marri, Ajinkya Kale, Jo Briggs, Chris Speed, Hailin Jin, Baldo Faieta, Alex Filipkowski, Zhe Lin, John Collomosse

ECCV 2022 · Vision-language and multimodal › Human and embodied modeling · approach: Composed and text-based image retrieval · 14 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
swap VGG-19 style encoder for ViT (ALADIN-ViT)ALADIN · BAM-FG-C5 style retrievalIR-1: 58.98 -> 64.48Improved
replace ALADIN-ViT embeddings with CLIP image encoder only, for tag generation (ablation)StyleBabel (FG) · StyleBabel tag generationWordNet score: 0.215 -> 0.352Improved
use image-level (FG) tags instead of cluster-level (coarse) (ablation)ALADIN-ViT · StyleBabel tag generationWordNet score: 0.225 -> 0.352Improved
use expert Grounded Theory annotation instead of mturk crowd annotation (ablation)ALADIN-ViT · StyleBabel tag generationWordNet score: 0.164 -> 0.352Improved
train on StyleBabel (coarse) instead of web-scale CLIP baselineCLIP · StyleBabel tag generationWordNet score: 0.168 -> 0.187Improved
collect tags via mturk instead of web-scale baseline (ablation)ALADIN-ViT · StyleBabel tag generationWordNet score: 0.168 -> 0.164No clear effect
train VirTex on StyleBabel image-level instead of MS-COCOVirTex · StyleBabel caption generationCIDEr: 0.022 -> 0.350Improved
add image-level fine-tuning to cluster-level model (CL+IL) (ablation)VirTex · StyleBabel caption generationCIDEr: 0.076 -> 0.372Improved
add image-level labels on top of IL-only training (CL+IL vs IL) (ablation)VirTex · StyleBabel caption generationCIDEr: 0.350 -> 0.372Improved
use VirTex instead of ALADIN-ViT LSTM for captioningStyleBabel (CL+IL) · StyleBabel caption generationCIDEr: 0.034 -> 0.372Improved
use ALADIN-ViT embeddings instead of ResNet with LSTM captioner (ablation)StyleBabel (CL+IL) · StyleBabel caption generationCIDEr: 0.017 -> 0.034Improved
use VirTex trained on StyleBabel instead of Artemis fine-tuned on StyleBabelVirTex · StyleBabel caption generationCIDEr: 0.029 -> 0.372Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy