Research map / Reinforcement learning / Visual and representation RL
Weak-to-Strong Generalization with Failure Trajectories
Ruimeng Ye, Zihan Wang, Yang Xiao, Zinan Ling, Manling Li, Bo Hui
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add tree-derived preference pairs instead of unstructured random pairs for DPO (ablation) | W2SG DPO, Llama3-8B · AlfWorld | average reward: 60.4 -> 61.9 | Improved |
| add MCTS over the trajectory tree instead of Tree DPO | W2SG, Llama-2-13B strong · WebShop | average reward: 53.2 -> 56.9 | Improved |
| use weak-model trajectory MCTS instead of plain SFT of the strong model | Llama-2-13B · ScienceWorld | average reward: 53.6 -> 58.2 | Improved |
| raise DPO beta from 0.1 to 0.5 (ablation) | W2SG Tree DPO (WTS) · ScienceWorld | average reward: 54.9 -> 49.2 | Hurt |
| raise MCTS tree breadth from 6 to 7 (ablation) | W2SG MCTS · ScienceWorld | average reward: 58.2 -> 54.9 | Hurt |
| raise MCTS tree breadth from 5 to 6 (ablation) | W2SG MCTS · ScienceWorld | average reward: 57.6 -> 58.2 | Improved |
| increase number of weak trajectories beyond optimal range (ablation) | W2SG · AlfWorld | average reward | Mixed |
| use Llama2-13B instead of Llama2-7B as weak model (ablation) | W2SG MCTS, Llama3-8B strong · WebShop | average reward: 63.7 -> 65.3 | Improved |
| use Llama2-13B instead of Llama2-7B as weak model (ablation) | W2SG Tree DPO, Llama3-8B strong · WebShop | average reward: 61.3 -> 62.0 | Improved |
| apply W2SG MCTS to Qwen2.5-14B strong with Qwen2.5-7B weak | Qwen2.5-14B · WebShop | average reward: 71.7 -> 76.1 | Improved |
| apply W2SG MCTS to Llama2-70B strong | Llama2-70B · ScienceWorld | average reward: 64.6 -> 69.1 | Improved |
| add best-of-8 sampling to SFT strong model | Llama-2-13B · WebShop | average reward: 51.0 -> 52.3 | Improved |
Closest papers by meaning
- The Mechanism of Weak-to-Strong Generalization: Feature Elicitation from Latent Knowledge (NeurIPS 2026 · similarity 0.90)
- Representations Shape Weak-to-Strong Generalization: Theoretical Insights and Empirical Predictions (ICML 2025 · similarity 0.89)
- Quantifying the Gain in Weak-to-Strong Generalization (NeurIPS 2024 · similarity 0.88)
- Strong Empowered and Aligned Weak Mastered Annotation for Weak-to-Strong Generalization (AAAI 2025 · similarity 0.87)
- Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision (ICML 2024 · similarity 0.87)
- Weak-to-Strong Generalization under Distribution Shifts (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
