Research map / Large language models / Language, safety and interpretability

IDEAFix: Evaluation Framework for Creative Defixation Prompting in LLMs

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: LLM unlearning and benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces IDEAFix, a compositional evaluation framework (81 briefs x attributes, 25 defixation prompts, 14,350 inputs) that measures fluency, diversity, novelty, rarity, overlap and perplexity of idea lists from five LLMs, finding attribute and prompt effects plus persistent cross-model homogenization.

Problem

LLM creativity evaluations on divergent thinking are either narrow and decontextualized or confound task, prompting and evaluation design, and the effect of structured defixation prompting on LLM idea generation is underexplored.

Approach

A controlled, compositional benchmark that crosses design briefs, typicality/valence attributes and creativity-method-inspired prompts (including AI-specific category-negation prompts), scored with embedding-based distance metrics for originality and cross-model homogenization.

Key results

• Llama-3.1-70B and Grok show highest diversity/novelty/rarity (e.g., Llama novelty 0.556 and rarity 0.294 vs Gemini 0.463 and 0.238), while Grok has highest fluency (20.16 ideas).
• All method-inspired prompts outperform control prompts on novelty/rarity significance tests; brainstorming and AI-specific prompts yield the strongest top-5 novelty gains.
• Persistent cross-model homogenization: solutions cluster in a shared semantic space dominated by digital and technological innovation.

Surprising findings

• Simpler prompts emphasizing explicit divergence cues (e.g., 'wild', 'unconventional') outperform complex method-based prompts such as C-K theory or SCAMPER.
• Classic literature briefs produce more ideas but lower novelty, diversity and rarity, suggesting convergence toward training-data exposure.

Limitations

• Metrics are embedding-based automatic proxies and not validated with human judgments of usefulness or originality.
• Prompting strategies cover only a subset of defixation methods and are non-iterative.
• Some sensitive briefs trigger refusals, and outputs were filtered by manual inspection rules.

Experimental setup

Datasets: IDEAFix briefs and attributes (constructed)

Benchmarks: IDEAFix

Metrics: fluency diversity novelty rarity overlap perplexity

Baselines: Control prompts (neutral brief)

Builds on: Alternative Uses Task Brainstorming C-K theory TRIZ SCAMPER

Closest papers by meaning

• Within-Model vs Between-Prompt Variability in Large Language Models for Creative Tasks (NeurIPS 2026 · similarity 0.89)
• CLAWS:Creativity detection for LLM-generated solutions using Attention Window of Sections (NeurIPS 2025 · similarity 0.87)
• MP: Endowing Large Language Models with Lateral Thinking (AAAI 2025 · similarity 0.86)
• Weak-eval-Strong: Evaluating and Eliciting Lateral Thinking of LLMs with Situation Puzzles (NeurIPS 2024 · similarity 0.86)
• Let's Think Outside the Box: Exploring Leap-of-Thought in Large Language Models with Creative Humor Generation (CVPR 2024 · similarity 0.86)
• CreBench: Human-Aligned Creativity Evaluation from Idea to Process to Product (AAAI 2026 · similarity 0.86)