Research map / Vision-language and multimodal / Human and embodied modeling
SemStyle: Learning to Generate Stylised Image Captions Using Unaligned Text
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Separate image semantics into terms and generate the target style from those terms | SemStyle · Romance-styled MSCOCO captions | SPICE: 0.144 vs 0.088 | Helped |
| Use the target-style term to generate romance-style captions | SemStyle · MSCOCO human style evaluation | story-like captions: 41.9% vs 6.2% | Helped |
| Randomize semantic term ordering (ablation) | SemStyle · Romance-styled MSCOCO captions | SPICE: 0.144 -> 0.134 | Hurt |
| Randomize semantic term ordering (ablation) | SemStyle · Romance-styled MSCOCO captions | CLF: 0.589 -> 0.501 | Hurt |
| Use raw words instead of POS-tagged, lemmatized, FrameNet terms (ablation) | SemStyle · Romance-styled MSCOCO captions | CLF: 0.589 -> 0.407 | Hurt |
| Use verb lemmas instead of FrameNet frames (ablation) | SemStyle · Romance-styled MSCOCO captions | CLF: 0.589 -> 0.533 | Hurt |
| Use verb lemmas instead of FrameNet frames (ablation) | SemStyle · Romance-styled MSCOCO captions | SPICE: 0.144 -> 0.148 | No clear effect |
| Train the language generator on romance text only rather than both romance and descriptive captions (ablation) | SemStyle · Romance-styled MSCOCO captions | SPICE: 0.144 -> 0.138 | Hurt |
| Train the language generator on romance text only rather than both romance and descriptive captions (ablation) | SemStyle · MSCOCO human relevance evaluation | clear and accurate captions: 43.8% -> 34.7% | Hurt |
| Use ground-truth ordered terms instead of randomized term order (ablation) | SemStyle term generator · MSCOCO | SPICE: 0.134 -> 0.144 | Helped |
| Use a descriptive target-style term at test time (ablation) | SemStyle · MSCOCO | SPICE: 0.154 -> 0.157 | No clear effect |
| Use scheduled sampling instead of teacher forcing during term-generator training (ablation) | SemStyle term generator · MSCOCO | term-generator performance | No clear effect |
Closest papers by meaning
- Panoptic Captioning: An Equivalence Bridge for Image and Text (NeurIPS 2025 · similarity 0.86)
- Elucidating the design space of language models for image generation (ICML 2025 · similarity 0.85)
- Patch Matters: Training-free Fine-grained Image Caption Enhancement via Local Perception (CVPR 2025 · similarity 0.85)
- Semantic and Expressive Variations in Image Captions Across Languages (CVPR 2025 · similarity 0.85)
- Revisit Large-Scale Image-Caption Data in Pre-training Multimodal Foundation Models (ICLR 2025 · similarity 0.84)
- Noise-Aware Image Captioning with Progressively Exploring Mismatched Words (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.