Research map / Large language models / Alignment and preferences
TO-GATE: Clarifying Questions and Summarizing Responses with Trajectory Optimization for Eliciting Human Preference
Yulin Dou, Jiangming Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add trajectory-optimized contrastive training (TO-GATE) over STaR-GATE | Mistral-7B questioner · Preference elicitation win rate vs M0 | win rate: 73.83 -> 83.15 | Improved |
| add trajectory-optimized contrastive training (TO-GATE) over STaR-GATE | Gemma-7B questioner · Preference elicitation win rate vs M0 | win rate: 77.76 -> 81.50 | Improved |
| plain DPO without SFT initialization instead of supervised STaR-GATE | questioner · Preference elicitation win rate vs M0 | win rate: 73.83 -> 61.33 (Mistral-7B) | Hurt |
| add the clarification resolver (DPO on trajectories) (ablation) | TO-GATE · Preference elicitation win rate vs M0 | win rate: 77.50 -> 83.15 | Improved |
| add the summarizer (separate response loss) (ablation) | TO-GATE · Preference elicitation win rate vs M0 | win rate: 81.00 -> 83.15 | Improved |
| increase response-loss weight lambda from 2 to 6 (ablation) | TO-GATE M1 · Preference elicitation win rate vs M0 | win rate: 78.66 -> 75.61 | Hurt |
| more training iterations: clarification vs response quality | TO-GATE · Preference elicitation (Figure 4) | clarification log-prob and response score | Mixed |
Closest papers by meaning
- Learning to Clarify: Multi-turn Conversations with Action-Based Contrastive Self-Training (ICLR 2025 · similarity 0.89)
- Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questions (ICLR 2025 · similarity 0.89)
- Eliciting Human Preferences with Language Models (ICLR 2025 · similarity 0.88)
- Optimal Design for Human Preference Elicitation (NeurIPS 2024 · similarity 0.86)
- Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs (ICLR 2025 · similarity 0.86)
- Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond) (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
