Research map / Large language models / Language, safety and interpretability
The Invisible Lottery: How Subtle Cues Steer Algorithm Choice in LLM Code Generation
Akanksha Narula, Mofasshara Binte Rafique, Laurent Bindschaedler
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add a readability instead of space constraint cue | LLM code generation, tree traversal · Tree traversal | recursive implementation rate: 0% -> 89% | Improved |
| use prototype instead of interview context | LLM code generation, expression parsing · Expression parsing | eval() usage: 6.1% -> 69.7% | Hurt |
| use academic instead of junior persona | LLM code generation, memoization · Memoization | pass rate: 100% -> 80.0% | Hurt |
| add import functools | LLM code generation, memoization · Memoization | top-down memoization rate: 17.1% -> 82.9% | Improved |
| add a space constraint cue | LLM code generation, rate limiter · Rate limiter | sliding-window rate: 90.0% -> 3.6% | Hurt |
| add semantically neutral placebo cues (team names, colors, ids) | LLM code generation · 5 API models | steering (pp): 26.1 mean pp shift | Improved |
| change surface typography of a time-critical cue, meaning held fixed (ablation) | memoization prompt · Memoization | matrix exponentiation rate: 0% baseline -> 14-41% | No clear effect |
| name the algorithm explicitly in the prompt | LLM code generation · 120 runs, 3 API models | pass rate: 120/120 pass; compliance 60.8% strict, 98.3% lenient | Improved |
| explicitly ask GPT-5 for matrix exponentiation instead of academic persona | GPT-5, memoization · Memoization | pass rate: 0/5 -> 5/5 | Improved |
| raise sampling temperature from 0.0 to 1.0 (ablation) | Qwen2.5-Coder 32B · Cue steering grid | mean capacity (pp): 49.7 -> 64.7 capacity; pass 93.0% -> 91.4% | Mixed |
| quantize DeepSeek-R1 70B from FP16 to Q8_0 (ablation) | DeepSeek-R1 70B · Cue steering grid | capacity and pass rate: -4.5 pp capacity, -1.6 pp pass rate | Mixed |
| raise classifier confidence threshold from 0.3 to 0.7 (ablation) | AST classifier · Tree traversal + constraint | steering delta (pp): 88.6 -> 88.6 | No clear effect |
Closest papers by meaning
- What Makes Large Language Models Reason in (Multi-Turn) Code Generation? (ICLR 2025 · similarity 0.87)
- Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting (ICLR 2024 · similarity 0.86)
- Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck (NeurIPS 2026 · similarity 0.86)
- The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs (ICLR 2026 · similarity 0.85)
- Intention Chain-of-Thought Prompting with Dynamic Routing for Code Generation (AAAI 2026 · similarity 0.85)
- Within-Model vs Between-Prompt Variability in Large Language Models for Creative Tasks (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
