Research map / Large language models / Retrieval and knowledge

Innovative Thinking, Infinite Humor: Humor Research of Large Language Models through Structured Thought Leaps

ICLR 2025 · Large language models › Retrieval and knowledge · approach: Explanatory and multiple-choice QA benchmarks · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LoL, a two-stage SFT+DPO framework that uses automatic instruction evolution with three agents and a teacher-student judgment loop, plus rationale-augmented online DPO (GESIT), to improve LLM humor judgment and generation.

Problem

LLMs struggle with multi-hop creative reasoning needed for humor generation and humor judgment, because the knowledge graph linking question and punchline entities is sparse and training data only captures question-answer endpoints.

Approach

Augments humor data with an automatic three-agent instruction-evolution pipeline (generator, imaginator, selector/critic) and a GPT-4o teacher-student prompt loop for judgment, then runs online DPO where a frozen judge model labels sampled responses and GPT-4o supplies causal rationales appended to preference pairs.

Key results

• On English SemEval 2021 2T1, LoL (Qwen1.5-32B based) reaches 96.58% accuracy versus 85.09% for GPT-4o and 80.05% for QwQ.
• On Chinese Ruozhiba-style 2T1, LoL scores 95.35% versus 76.40% for GPT-4o and 90.80% for QwQ.
• Human study with about 3000 votes shows users preferred LoL outputs for creativity and humor over GPT-4o, Qwen1.5-32B and CLoT.

Measured effects reported in the paper

Surprising findings

• Adding the Oogiri-GO-only AIE data caused a slight performance drop on the English SemEval benchmarks, which the authors attribute to overfitting to divergent thinking.
• GPT-4o and other strong LLMs perform near chance on several hard humor judgment tasks, e.g., 3T1 and 4T1 SemEval 2021 accuracy around 30-43%.

Limitations

• Creativity of training data is uneven, so generated humor quality depends on annotator preferences.
• Evaluation relies on small human study (15 questionnaires) and GPT-4-based judgments of logical coherence.
• Several reported results are internally inconsistent in naming (LoL, CLoST, GESIT, DIET), making the method description hard to verify.

Experimental setup

Datasets: Oogiri-GO SemEval 2021 Task 7 SemEval 2020 Task 7 Chinese community humor data (Ruozhiba) Chinese Word Vectors

Benchmarks: SemEval 2021 Task 7 SemEval 2020 Task 7 Oogiri-GO Ruozhiba Chinese humor benchmark

Metrics: top-1 accuracy DAT score human win rate logical coherence rating

Baselines: GPT-4o CLoT QwQ Qwen1.5-32B LLaMA3-70B Qwen2.5-32B Qwen2-57B

Builds on: Creative Leap of Thought (CLoT) Evol-Instruct Direct Preference Optimization LoRA

Closest papers by meaning

• Let's Think Outside the Box: Exploring Leap-of-Thought in Large Language Models with Creative Humor Generation (CVPR 2024 · similarity 0.91)
• HUMORCHAIN: Theory-Guided Multi-Stage Reasoning for Interpretable Multimodal Humor Generation (CVPR 2026 · similarity 0.89)
• Cracking the Code of Juxtaposition: Can AI Models Understand the Humorous Contradictions (NeurIPS 2024 · similarity 0.87)
• From Perception to Punchline: Empowering VLM with the Art of In-the-wild Memes (NeurIPS 2026 · similarity 0.86)
• Talk Funny! A Large-Scale Humor Response Dataset with Chain-of-Humor Interpretation (AAAI 2024 · similarity 0.86)
• Flow of Reasoning: Training LLMs for Divergent Reasoning with Minimal Examples (ICML 2025 · similarity 0.85)