Research map / Large language models / Alignment and preferences
Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Releases a New Yorker cartoon caption preference dataset (250M+ crowd ratings on 2.2M captions over 365 contests) and benchmarks LLM/MLLM humor generation with group-comparison GPT-4 judging, comparing SFT, RLHF (PPO), DPO and Best-of-N on Mistral-7B and LLaVA.
Problem
Aligning LLMs and MLLMs to generate funny captions for creative tasks, where humor is subjective and preference signals are noisy; existing alignment methods and evaluations are mostly tuned for helpfulness and harmlessness.
Approach
Uses crowdsourced ratings from The New Yorker's weekly caption contest as a large human preference dataset, and introduces group-based evaluation (ten captions per group, compared against human rank groups) with GPT-4 judges, plus a calibrated decision threshold to debias A/B position preference.
Key results
- Claude-3-Opus achieves 54.40% overall win rate vs human Top 10 captions and 88.46% vs contestant median under GPT-4 group comparison, while the best pick win rate vs Top 10 is 47.80% for GPT-4o Vision.
- Human expert preferred Claude-3-Opus captions over human top-10 only 1.6% of the time, versus 35.4% for crowd workers.
- Mistral-7B DPO is the best open-source model at 9.34% overall win rate vs Top 10 and 10.44% best pick; SFT lowers performance while RLHF (PPO) does not outperform BoN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply DPO fine-tuning instead of zero-shot generation (ablation) | Mistral-7B Instruct · Top 10 (Best Pick) | win rate: 1.65 -> 10.44 | Helped |
| Apply DPO fine-tuning instead of zero-shot generation (ablation) | Mistral-7B Instruct · Top 10 (Overall) | win rate: 4.95 -> 9.34 | Helped |
| Apply SFT fine-tuning instead of zero-shot generation (ablation) | Mistral-7B Instruct · Top 10 (Overall) | win rate: 4.95 -> 3.85 | Hurt |
| Apply SFT fine-tuning instead of zero-shot generation (ablation) | Mistral-7B Instruct · Top 10 (Best Pick) | win rate: 1.65 -> 0.55 | Hurt |
| Use Best-of-N sampling instead of zero-shot sampling (ablation) | Mistral-7B Instruct · Overall win rate across comparison groups | win rate: 4.95 -> 6.59; 8.79 -> 16.48; 11.54 -> 21.43; 25.82 -> 35.71 | Helped |
| Use Best-of-N sampling instead of zero-shot sampling (ablation) | Mistral-7B Instruct · Best Pick win rate across comparison groups | win rate: 1.65 -> 1.65; 1.65 -> 2.20; 3.85 -> 3.30; 12.64 -> 10.44 | Mixed |
| Use RLHF instead of zero-shot generation (ablation) | Mistral-7B Instruct · Overall win rate across comparison groups | win rate: 4.95 -> 8.79; 8.79 -> 9.34; 11.54 -> 11.54; 25.82 -> 24.73 | Mixed |
| Provide cartoon images to the evaluator in addition to text descriptions | GPT4-Turbo-vision evaluator · Top 10 vs #1000–#1009 (pairwise ranking) | ranking accuracy: 67% -> 60.5% | Hurt |
| Use GPT-4o descriptions without cartoon images rather than GPT-4o-vision with cartoon images | GPT-4o generation · Overall win rate across human ranking groups | win rate | Helped |
| Use the multimodal LLaVA model rather than language-only Mistral | Caption generation · Cartoon caption generation | win rate | Hurt |
| Use DPO rather than SFT or PPO (ablation) | Mistral caption generation · Caption diversity | diversity | Helped |
| Use PPO training iterations that increase reward and reduce loss (ablation) | Mistral-7B RLHF · Contestant Median (Overall) | win rate: 24.73 -> 4.95 | Hurt |
Surprising findings
- Adding the cartoon image to GPT-4 judges reduced evaluator accuracy compared with text descriptions alone.
- Language-only Mistral outperformed the multimodal LLaVA on caption win rates.
Limitations
- Evaluation relies on GPT-4 judges whose bias needed threshold recalibration; human expert evaluation is limited to one editor.
- RLHF with PPO failed to improve humor win rates despite reward gains, suggesting reward model or distribution-shift issues.
Experimental setup
Datasets: New Yorker Caption Contest ratings (365 contests) cartoon description dataset from prior work
Benchmarks: HumorousAI group comparison (overall) HumorousAI group comparison (best pick) New Yorker caption contest held-out cartoons
Metrics: win rate ranking accuracy expectation-adjusted distinct n-grams SBERT similarity preference rate
Baselines: GPT-3.5 Turbo GPT-4o GPT-4o Vision Claude-3-Opus LLaVA Mistral-7B SFT Mistral-7B RLHF Mistral-7B DPO Mistral-7B BoN
Builds on: DPO RLHF PPO LoRA UCB
Closest papers by meaning
- From Perception to Punchline: Empowering VLM with the Art of In-the-wild Memes (NeurIPS 2026 · similarity 0.90)
- HUMORCHAIN: Theory-Guided Multi-Stage Reasoning for Interpretable Multimodal Humor Generation (CVPR 2026 · similarity 0.90)
- Cracking the Code of Juxtaposition: Can AI Models Understand the Humorous Contradictions (NeurIPS 2024 · similarity 0.88)
- Talk Funny! A Large-Scale Humor Response Dataset with Chain-of-Humor Interpretation (AAAI 2024 · similarity 0.85)
- Innovative Thinking, Infinite Humor: Humor Research of Large Language Models through Structured Thought Leaps (ICLR 2025 · similarity 0.85)
- CaptionQA: Is Your Caption as Useful as the Image Itself? (CVPR 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.