Research map / Reinforcement learning / RL from feedback
Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-control
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add task-specific music rewards while retaining the MLE prior | Sequence Tutor Q · Melody generation | Repeated notes: 63.3% -> 0.0% | Helped |
| Add task-specific music rewards while retaining the MLE prior | Sequence Tutor Q · Melody generation | Starts with tonic: 0.9% -> 28.8% | Helped |
| Add task-specific music rewards while retaining the MLE prior | Sequence Tutor Q · Melody generation | Leaps resolved: 77.2% -> 91.1% | Helped |
| Add task-specific music rewards while retaining the MLE prior | Sequence Tutor Q · Melody generation | Notes in motif: 5.9% -> 75.7% | Helped |
| Use G-learning instead of Q- or Ψ-learning | Sequence Tutor · Melody generation | Mean autocorr. lag 1: -.16 (MLE), -.11 (Q), -.10 (Ψ), .55 (G) | Mixed |
| Apply KL-control and music-theory rewards rather than MLE alone | Sequence Tutor Q, Ψ, and G · Melody generation | User preference: p < .001 | Helped |
| Use Sequence Tutor Q instead of MLE | Molecule generation · Molecule generation | Percent valid: 30.3% -> 35.8% | Helped |
| Use Sequence Tutor Q instead of MLE | Molecule generation · Molecule generation | Mean logP: 2.07 -> 4.21 | Helped |
| Use Sequence Tutor Q instead of MLE | Molecule generation · Molecule generation | Mean QED: .678 -> .417 | Hurt |
| Use Sequence Tutor Q instead of MLE | Molecule generation · Molecule generation | Mean SA penalty: -2.77 -> -1.79 | Helped |
Closest papers by meaning
- ESRL: Efficient Sampling-Based Reinforcement Learning for Sequence Generation (AAAI 2024 · similarity 0.86)
- Parallelizing Linear Recurrent Neural Nets Over Sequence Length (ICLR 2018 · similarity 0.85)
- SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient (AAAI 2017 · similarity 0.85)
- Recurrently Controlled Recurrent Networks (NeurIPS 2018 · similarity 0.85)
- Sequence to Sequence Reward Modeling: Improving RLHF by Language Feedback (AAAI 2025 · similarity 0.84)
- Improving Bilinear RNN with Closed-loop Control (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.