Research map / Large language models / Agents, code and math

Failing to Falsify: Evaluating and Mitigating Confirmation Bias in Language Models

NeurIPS 2026 · Large language models › Agents, code and math · approach: Long-horizon LLM agents · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Adapts Wason's 2-4-6 rule-discovery task to measure confirmation bias in 11 LLMs, shows prompt interventions (Dual-Goal, Think-in-Opposites) improve rule discovery, and distills Think-in-Opposites behavior into Qwen3 students that partially transfer to the Blicket test.

Problem

Whether LLMs exhibit confirmation bias when exploring hypotheses interactively (selecting evidence that confirms rather than falsifies their current guess), and whether it hurts rule discovery.

Approach

Measure confirmation bias in LLM agents as the incompatible-to-compatible (I:C) ratio of proposed tests in a multi-turn rule-discovery game, port human debiasing interventions to prompts, and internalize them via symbolic knowledge distillation of teacher test-turn outputs.

Key results

• Thinking-mode Qwen3-32B task success rises from 0.41 (baseline) to 0.61 with Think-in-Opposites on the Wason task.
• Intervention distillation Qwen3-32B + 32B TiO reaches 0.64 task success with I:C 0.33 (significant), vs. 0.41 and 0.14 for the base model.
• Distilled Qwen3-32B (Wason-trained TiO) on Blicket reaches 0.77 task success and I:C 0.88 vs 0.57 and 0.54 base.

Surprising findings

• Thinking-mode improvements in task success are not accompanied by consistent increases in I:C, suggesting gains are not driven by reduced confirmation bias.
• Dual-Goal prompting reduced task success for several non-thinking models (e.g., Qwen3-14B from 0.23 to 0.13).

Limitations

• Task deviates from the human Wason study (forced alternation between guess and test turns, no feedback on guesses), limiting direct comparison.
• Small test set (80 Wason episodes, 4 test rule groups) and only a handful of rule families, so statistical power is limited.
• Generalization to Blicket transfer is partial; distilled models do not match TiO prompting in the new domain.

Experimental setup

Datasets: LLM-generated rule sets (synthetic, 10 groups) Large Dataset of Generalization Patterns in the Number Game (human rules) Blicket test environment

Benchmarks: Wason 2-4-6 rule-discovery task (test split, 80 episodes) Blicket test (192 episodes)

Metrics: task success turns until success tokens per turn incompatible:compatible ratio

Baselines: Baseline prompt Think-in-Opposites Dual-Goal GPT-4o Gemini-2.5-Pro QwQ-32B DeepSeek-R1-Distill-Llama-70B Llama-3.3-70B-Instruct

Builds on: Wason 2-4-6 task Dual-Goal Think-in-Opposites symbolic knowledge distillation

Closest papers by meaning

• Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement (ICLR 2024 · similarity 0.89)
• LLM Agents Can Be Choice-Supportive Biased Evaluators: An Empirical Study (AAAI 2025 · similarity 0.88)
• Doing Experiments and Revising Rules with Natural Language and Probabilistic Reasoning (NeurIPS 2024 · similarity 0.87)
• Walk the Talk? Measuring the Faithfulness of Large Language Model Explanations (ICLR 2025 · similarity 0.87)
• Trust, but Don’t Verify: Epistemic Blind Spots in LLM Source Evaluation (NeurIPS 2026 · similarity 0.87)
• Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge (ICLR 2025 · similarity 0.87)