Research map / Large language models / Reasoning and chain of thought
Let's Think Outside the Box: Exploring Leap-of-Thought in Large Language Models with Creative Humor Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Creative Leap-of-Thought (CLoT), a two-stage LoRA-based training method (associable instruction tuning plus explorative self-refinement) that improves LLMs' non-sequential creative humor generation, evaluated on a new multilingual Oogiri-GO dataset.
Problem
Chain-of-thought reasoning is sequential and does not promote creative, out-of-the-box leaps of association; existing LLMs show insufficient leap-of-thought (LoT) ability, measured here via Oogiri-style humor generation.
Approach
Trains LoRA on LoT-oriented instruction templates built from Oogiri data (generation with random noun conditions and discrimination via selection/ranking), then self-refines by generating candidates under weakly-associated conditions, ranking/selecting the best, and retraining on them.
Key results
- Qwen-VL+CLoT improves average choice/ranking score over Qwen-VL by 9.1, 10.4, and 8.2 points on IT2T, I2T, and T2T multilingual tasks.
- CogVLM-17B+CLoT raises average I2T score from 29.9 to 51.6 on English Oogiri.
- CLoT-integrated Qwen-VL improves over vanilla Qwen-VL by about 8% on the Cloud Guessing Game and 5% on the Divergent Association Task.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| associable instruction tuning (LoRA on Oogiri-GO LoT instructions) (ablation) | Qwen-VL 7B · Oogiri-GO IT2T/I2T/T2T multilingual | avg accuracy/NDCG: +6.8, +6.7, +4.3 | Helped |
| adding explorative self-refinement on top of AIT (full CLoT) | Qwen-VL 7B · Oogiri-GO IT2T/I2T/T2T multilingual | avg accuracy/NDCG: +9.1, +10.4, +8.2 over vanilla Qwen-VL | Helped |
| associable instruction tuning (AIT) (ablation) | CogVLM-17B · Oogiri-GO English I2T and T2T | avg accuracy/NDCG: +13.1 (I2T), +10.3 (T2T) | Helped |
| full CLoT (AIT + explorative self-refinement) | CogVLM-17B · Oogiri-GO English I2T and T2T | avg accuracy/NDCG: +21.7 (I2T), +16.4 (T2T) | Helped |
| weakly-associated (random noun) conditions instead of strongly-associated (current image caption noun) conditions during explorative remote association (ablation) | CLoT self-refinement data generation (CogVLM, English I2T) · Oogiri-GO English I2T | accuracy/NDCG (figure only) | Helped |
| one round of self-refinement vs additional rounds (ablation) | CLoT self-refinement · Oogiri-GO English I2T (CogVLM) | accuracy/NDCG (figure only) | No clear effect |
| prompting-based CoT, CoT-SC, and PLoT ('let's think outside the box') vs CLoT | Qwen-VL multilingual I2T · Oogiri-GO multilingual I2T | accuracy/NDCG | No clear effect |
| CLoT applied to Qwen-VL on cloud guessing game and divergent association task | Qwen-VL 7B · Cloud Guessing Game and DAT | accuracy / ASD: about 8% on CGG and 5% on DAT | Helped |
| CLoT (text-only generalization tested on English T2T) | Qwen-VL and CogVLM-17B vs text-only LLMs · Oogiri-GO English T2T | avg accuracy/NDCG | Helped |
Surprising findings
- CoT sometimes impairs LoT performance of Qwen-VL on Oogiri, and CoT-style frameworks do not improve LoT.
- LoT cannot be elicited by prompting alone (e.g., 'let's think outside the box'); it requires training with CLoT.
Limitations
- Humor evaluation is subjective; creativity measured via proxy choice/ranking questions built from human likes.
- Gains from a second round of self-refinement are not significant; expanding the noun condition set is left to future work.
- Oogiri data is sourced from online platforms with cultural and translation constraints (e.g., no English IT2T data).
Experimental setup
Datasets: Oogiri-GO Bokete Twitter Weibo
Benchmarks: Oogiri-GO test set (choice and ranking questions) Cloud Guessing Game Divergent Association Task
Metrics: accuracy NDCG top-1 accuracy average semantic distance user vote percentage
Baselines: GPT-4V LLaVA-1.5 MiniGPT-v2 mPLUG-Owl VisualGLM-6B Qwen-VL CogVLM-17B GPT-3.5 GPT-4 LLaMA2 Baichuan2 Qwen
Builds on: chain-of-thought LoRA Qwen-VL CogVLM
Closest papers by meaning
- Innovative Thinking, Infinite Humor: Humor Research of Large Language Models through Structured Thought Leaps (ICLR 2025 · similarity 0.91)
- Weak-eval-Strong: Evaluating and Eliciting Lateral Thinking of LLMs with Situation Puzzles (NeurIPS 2024 · similarity 0.88)
- GameArena: Evaluating LLM Reasoning through Live Computer Games (ICLR 2025 · similarity 0.87)
- CLAWS:Creativity detection for LLM-generated solutions using Attention Window of Sections (NeurIPS 2025 · similarity 0.87)
- HUMORCHAIN: Theory-Guided Multi-Stage Reasoning for Interpretable Multimodal Humor Generation (CVPR 2026 · similarity 0.86)
- Improving Rationality in the Reasoning Process of Language Models through Self-playing Game (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.