Research map / Label-efficient and robust learning / Knowledge distillation
Student-Informed Teacher Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Student-Informed Teacher Training, which jointly trains a privileged teacher and a student by penalizing teacher-student action mismatch in the teacher's reward and adding a KL gradient to the teacher's policy update, plus an alignment phase using a proxy student.
Problem
Privileged teacher-student imitation learning suffers from information asymmetry: the teacher ignores the student's limited observations and partial observability, so its actions cannot be imitated by the student.
Approach
Derive from the imitation performance bound an objective that adds the teacher-student action KL divergence as a reward penalty and as a direct loss on the teacher, using a cheap proxy student network to predict student actions from teacher observations, and alternate this with a supervised teacher-student feature alignment phase.
Key results
- Quadrotor obstacle avoidance: success rate 0.46 with alignment vs 0.08 (DAgger) and 0.05 (BC); surpasses DWBC by 0.11, HLRL by 0.15, COSIL by 0.16.
- Drawer opening: student success up to 0.77 vs 0.47 (DAgger) and 0.27 (BC); ablation with lambda_1=0.025 reaches 0.95 +/- 0.03 success.
- Color maze: student trained with the framework reaches the target in all test runs, while BC, DAgger and non-aligned students fail on lava-path ambiguity.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train the teacher with the alignment framework | Student-Informed Teacher Training · Quadrotor obstacle avoidance | student success rate: 0.08 -> 0.46 | Helped |
| train the teacher with the alignment framework | Student-Informed Teacher Training · Vision-based manipulation | student success rate: 0.47 -> 0.77 | Helped |
| add the KL-Divergence gradient to the policy update (ablation) | Student-Informed Teacher Training · Vision-based manipulation | success rate: 0.47 ± 0.37 -> 0.88 ± 0.07 | Helped |
| add the alignment penalty to the reward (ablation) | Student-Informed Teacher Training · Vision-based manipulation | success rate: 0.74 ± 0.08 -> 0.88 ± 0.07 | Helped |
| use a shared action decoder (ablation) | Student-Informed Teacher Training · Vision-based manipulation | success rate: 0.62 ± 0.27 -> 0.88 ± 0.07 | Helped |
| set alignment penalty coefficient λ₁ to 0.1 (ablation) | Student-Informed Teacher Training · Vision-based manipulation | success rate: 0.88 ± 0.07 -> 0.81 ± 0.20 | Hurt |
| set alignment penalty coefficient λ₁ to 0.075 (ablation) | Student-Informed Teacher Training · Vision-based manipulation | success rate: 0.88 ± 0.07 -> 0.77 ± 0.18 | Hurt |
| set alignment penalty coefficient λ₁ to 0.025 (ablation) | Student-Informed Teacher Training · Vision-based manipulation | success rate: 0.88 ± 0.07 -> 0.95 ± 0.03 | Helped |
| train with alignment rather than without alignment (ablation) | Student-Informed Teacher Training · Quadrotor obstacle avoidance | velocity angle (degrees): 63.2 -> 32.2 | Helped |
| train with alignment rather than without alignment (ablation) | Student-Informed Teacher Training · Quadrotor obstacle avoidance | number of obstacles in view: 2.61 -> 3.51 | Helped |
Surprising findings
- Removing the alignment penalty in the manipulation task causes only a 14% performance drop, while the shared action decoder and KL gradient are reported as crucial.
- Baseline students achieve non-zero success mainly due to favorable initializations or small cabinet position sampling intervals, which allow memorization without relying on images.
Limitations
- Alignment requires paired teacher and student observations on a subset of data, which needs rendering of student images.
- Requires a shared action decoder and state-independent Gaussian policy variance assumption for the simplified KL.
- Evaluated on a small number of simulated tasks with several hand-tuned loss weights.
Experimental setup
Benchmarks: Color Maze Vision-based quadrotor obstacle avoidance (custom Flightmare env) Vision-based drawer opening (Omniverse Isaac Gym Franka cabinet)
Metrics: success rate return velocity angle number of obstacles in view
Baselines: Behavior Cloning DAgger HLRL DWBC COSIL w/o Alignment
Builds on: PPO DAgger behavior cloning DINOv2 Flightmare
Closest papers by meaning
- RAIL: Representation-Aligned Imitation Learning for Student-Compatible Teacher Policies (NeurIPS 2026 · similarity 0.92)
- Coaching a Teachable Student (CVPR 2023 · similarity 0.89)
- Mimicking the Maestro: Exploring the Efficacy of a Virtual AI Teacher in Fine Motor Skill Acquisition (AAAI 2024 · similarity 0.86)
- Bridging the Imitation Gap by Adaptive Insubordination (NeurIPS 2021 · similarity 0.86)
- Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning (NeurIPS 2024 · similarity 0.85)
- Multi-Teacher Knowledge Distillation with Reinforcement Learning for Visual Recognition (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.