Research map / Large language models / Reasoning and chain of thought

Privileged Information Distillation for Language Models

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes π-Distill and On-Policy Self-Distillation (OPSD), which share parameters between a privileged-information (PI)-conditioned teacher and an unconditioned student to distill frontier-agent behavior from actions alone, without chain-of-thought.

Problem

Transferring capabilities learned with training-time privileged information (PI) to a test-time policy that lacks PI, specifically distilling frontier multi-turn agents when their chain-of-thought traces are hidden.

Approach

A single shared-parameter model acts as a PI-conditioned teacher and an unconditioned student, trained jointly with a weighted teacher/student objective (π-Distill), or with an on-policy reverse-KL self-distillation penalty toward the PI-conditioned teacher (OPSD).

Key results

• On Qwen3-8B, pi-Distill (alpha=1) reaches 44.1% on Travel Planner vs 32.3% for SFT w/ CoT + RL.
• On Qwen3-8B tau-Bench Retail, pi-Distill (alpha=0) reaches 31.1% vs 29.1% for SFT w/ CoT + RL.
• On Qwen3-4B Travel Planner, pi-Distill (alpha=0.5) reaches 33.8% vs 26.4% for SFT w/ CoT + RL.

Surprising findings

• Standard RL consistently degrades performance relative to the base model on GEM OOD tasks.
• On Qwen3-4B, OPSD shows significant OOD degradation while pi-Distill does not, suggesting OPSD overfits teacher supervision on smaller models.

Limitations

• Experiments limited to models with 8B parameters or fewer.
• PI is derived from frontier model traces; obtaining PI without frontier actions or ground-truth answers is not addressed.
• Analysis is observational and does not systematically control variables.

Experimental setup

Datasets: tau-Bench retail (training tasks) Travel Planner (training tasks) GEM QA search suite (2Wiki, PopQA, TriviaQA, HotpotQA, Bamboogle, NaturalQuestions, Musique)

Benchmarks: Travel Planner tau-Bench Retail tau-Bench Airline (OOD) GEM QA search suite (OOD)

Metrics: pass rate / success rate Pass@1 Pass@10

Baselines: RL (GRPO) SFT w/ CoT SFT w/o CoT SFT w/ CoT + RL SFT w/o CoT + RL R1-Distill-Llama-8B base

Builds on: GRPO on-policy distillation STaR variational reasoners context distillation

Closest papers by meaning

• Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents (NeurIPS 2026 · similarity 0.90)
• Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models (ICML 2026 · similarity 0.88)
• Learning from Language Feedback via Variational Policy Distillation (NeurIPS 2026 · similarity 0.87)
• The Many Faces of On-Policy Distillation: Pitfalls, Mechanisms, and Fixes (NeurIPS 2026 · similarity 0.87)
• KL for a KL: On-Policy Distillation with Control Variate Baseline (NeurIPS 2026 · similarity 0.87)
• Learning Agentic Policy from Action Guidance (NeurIPS 2026 · similarity 0.86)