Research map / Vision-language and multimodal / Human and embodied modeling
StyleBabel: Artistic Style Tagging and Captioning
Dan Ruta, Andrew Gilbert, Pranav Aggarwal, Naveen Marri, Ajinkya Kale, Jo Briggs, Chris Speed, Hailin Jin, Baldo Faieta, Alex Filipkowski, Zhe Lin, John Collomosse
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| swap VGG-19 style encoder for ViT (ALADIN-ViT) | ALADIN · BAM-FG-C5 style retrieval | IR-1: 58.98 -> 64.48 | Improved |
| replace ALADIN-ViT embeddings with CLIP image encoder only, for tag generation (ablation) | StyleBabel (FG) · StyleBabel tag generation | WordNet score: 0.215 -> 0.352 | Improved |
| use image-level (FG) tags instead of cluster-level (coarse) (ablation) | ALADIN-ViT · StyleBabel tag generation | WordNet score: 0.225 -> 0.352 | Improved |
| use expert Grounded Theory annotation instead of mturk crowd annotation (ablation) | ALADIN-ViT · StyleBabel tag generation | WordNet score: 0.164 -> 0.352 | Improved |
| train on StyleBabel (coarse) instead of web-scale CLIP baseline | CLIP · StyleBabel tag generation | WordNet score: 0.168 -> 0.187 | Improved |
| collect tags via mturk instead of web-scale baseline (ablation) | ALADIN-ViT · StyleBabel tag generation | WordNet score: 0.168 -> 0.164 | No clear effect |
| train VirTex on StyleBabel image-level instead of MS-COCO | VirTex · StyleBabel caption generation | CIDEr: 0.022 -> 0.350 | Improved |
| add image-level fine-tuning to cluster-level model (CL+IL) (ablation) | VirTex · StyleBabel caption generation | CIDEr: 0.076 -> 0.372 | Improved |
| add image-level labels on top of IL-only training (CL+IL vs IL) (ablation) | VirTex · StyleBabel caption generation | CIDEr: 0.350 -> 0.372 | Improved |
| use VirTex instead of ALADIN-ViT LSTM for captioning | StyleBabel (CL+IL) · StyleBabel caption generation | CIDEr: 0.034 -> 0.372 | Improved |
| use ALADIN-ViT embeddings instead of ResNet with LSTM captioner (ablation) | StyleBabel (CL+IL) · StyleBabel caption generation | CIDEr: 0.017 -> 0.034 | Improved |
| use VirTex trained on StyleBabel instead of Artemis fine-tuned on StyleBabel | VirTex · StyleBabel caption generation | CIDEr: 0.029 -> 0.372 | Improved |
Closest papers by meaning
- ALADIN: All Layer Adaptive Instance Normalization for Fine-Grained Style Similarity (ICCV 2021 · similarity 0.89)
- Stylebreeder: Exploring and Democratizing Artistic Styles through Text-to-Image Models (NeurIPS 2024 · similarity 0.89)
- Investigating Style Similarity in Diffusion Models (ECCV 2024 · similarity 0.88)
- IntroStyle: Training-Free Introspective Style Attribution using Diffusion Features (ICCV 2025 · similarity 0.87)
- Artistic Style Discovery With Independent Components (CVPR 2022 · similarity 0.87)
- Alchemist: Turning Public Text-to-Image Data into Generative Gold (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
