Research map / Large language models / AI and society
Can LLMs Learn by Teaching for Better Reasoning? A Preliminary Study
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes learning-by-teaching (LbT) for LLMs, scoring teacher rationales by how well student models solve exam problems after in-context teaching, and uses this for answer selection (M1), DPO fine-tuning (M2), and iterative exemplar refinement with multiple students (M3).
Problem
Whether LLMs can improve their own reasoning by teaching other models, analogous to the learning-by-teaching effect in human education, rather than relying only on human data or stronger teachers.
Approach
Scores each teacher-generated rationale-answer pair by the exam accuracy of student LLMs that learn from it via in-context learning on similar problems (LbT score), and uses that signal for answer selection, preference fine-tuning, and iterative prompt/exemplar refinement with diverse students.
Key results
- On 181 MATH test problems with 256 TR-TA pairs, M1 (SUM) lifts GPT-4o (teacher and student GPT-4o mini) from 91.71% SC to 96.69%, and GPT-4o reaches 96.69% overall, a 3.32%-4.98% gain over SC.
- M1 with 24 TR-TA pairs beats SC with 256 pairs by 0.17%-8.29% accuracy on MATH at comparable or lower compute.
- M2 fine-tuning LLaMA3-8B with LbT-scored DPO reaches 32.2% on 500 MATH test problems versus 30.4% for correctness-DPO and 29.0% original.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use M1 student-feedback scoring and answer selection instead of self-consistency | GPT-4o on MATH · MATH 181 test problems, 256 TR-TA pairs | accuracy: 91.71 -> 96.69 | Helped |
| Use M1 student-feedback scoring and answer selection instead of self-consistency | LLaMA3-8B on MATH · MATH 181 test problems, 256 TR-TA pairs | accuracy: 64.64 -> 82.87 | Helped |
| Use student-feedback rationale scores rather than correctness scores for DPO | LLaMA3-8B · MATH 500 test problems | accuracy: 30.4 -> 32.2 | Helped |
| Use a different, weaker LLaMA3-8B student instead of the LLaMA3-70B teacher as its own student | M3 with LLaMA3-70B teacher · Liar dev and test combined, T=5 | F1: 63.96±1.19 -> 65.41±0.75 | Helped |
| Use both LLaMA3-8B and Mistral-7B students instead of the LLaMA3-70B teacher alone | M3 with LLaMA3-70B teacher · Liar dev and test combined, T=5 | F1: 63.96±1.19 -> 67.09±0.56 | Helped |
| Use M1 student-feedback selection rather than average sampling | LLaMA3-8B on LeetCode Game Theory · Game Theory, SG-1 | S-score: 0.215 -> 0.630 | Helped |
| Use M1 student-feedback selection rather than average sampling | LLaMA3-8B on LeetCode Game Theory · Game Theory, PW | S-score: 0.609 -> 0.508 | Hurt |
| Select by summed student-feedback scores rather than the highest individual score (ablation) | M1 on MATH with GPT-4o teaching GPT-4o mini · MATH 181 test problems, 256 TR-TA pairs | accuracy: 95.03 -> 96.69 | Helped |
| Use 24 M1 TR-TA pairs instead of 256 SC TR-TA pairs | LLaMA3-8B self-teaching · MATH 181 test problems | accuracy: 64.64 -> 72.93±2.15 | Helped |
| Score rationales using student feedback rather than self-evaluation | M1 on GPT-4 · MATH 70 problems, 16 TR-TA pairs | accuracy: 68.57 -> 72.86 | Helped |
| Add self-debugging to the student and M1 pipeline (ablation) | LLaMA3-8B on LeetCode General-1D · General-1D, LVP, S-score | S-score: 0.719 -> 0.719 | Helped |
| Add self-debugging to the student and M1 pipeline (ablation) | LLaMA3-8B on LeetCode Game Theory · Game Theory, SG-3 | S-score: 0.570 -> 0.570 | Helped |
Surprising findings
- Teaching with rationales (not labels) makes ICL following reliable, so teacher answer correctness correlates with student exam accuracy despite prior findings that ICL labels matter little.
- Weaker students (e.g., LLaMA3-8B or Mistral-7B) can help stronger teachers improve, supporting a weak-to-strong effect.
Limitations
- M1 and M2 require similar exam problems to the teaching problem, which were selected using human-provided dataset information.
- LbT-based scoring adds substantial additional inference cost.
- Code-synthesis gains are mixed and depend on teacher-student being from the same model family.
Experimental setup
Datasets: MATH MATH functional variants (181 test problems with 3 variants each) LeetCode Grandmaster DP study plan Liar Logical Fallacy
Benchmarks: MATH LeetCode Grandmaster DP study plan Liar Logical Fallacy
Metrics: accuracy S-score (hidden test pass rate) F1
Baselines: self-consistency greedy decoding self-evaluation scoring correctness-DPO Self-Debugging
Builds on: self-consistency direct preference optimization Reflexion Self-Debugging
Closest papers by meaning
- Teaching LLMs How to Learn with Contextual Fine-Tuning (ICLR 2025 · similarity 0.89)
- At Which Training Stage Does Code Data Help LLMs Reasoning? (ICLR 2024 · similarity 0.86)
- Thinking LLMs: General Instruction Following with Thought Generation (ICML 2025 · similarity 0.86)
- SuperCorrect: Advancing Small LLM Reasoning with Thought Template Distillation and Self-Correction (ICLR 2025 · similarity 0.86)
- Teaching Models to Teach Themselves: Reasoning at the Edge of Learnability (ICML 2026 · similarity 0.86)
- LLMs in the Classroom: Outcomes and Perceptions of Questions Written with the Aid of AI (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.