Research map / Large language models / Agents, code and math
How2Everything: Mining the Web for How-to Procedures to Evaluate and Improve LLMs
Yapei Chang, Kyle Lo, Mohit Iyyer, Luca Soldaini
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| RL with HOW2SCORE reward on Qwen3-4B-Instruct | Qwen3-4B-Instruct · HOW2BENCH | HOW2SCORE: 30.29 -> 43.52 | Improved |
| RL with HOW2SCORE reward on Qwen3-8B-Instruct | Qwen3-8B-Instruct · HOW2BENCH | HOW2SCORE: 38.52 -> 48.62 | Improved |
| RL with HOW2SCORE reward on Olmo-3-7B-Think | Olmo-3-7B-Think · HOW2BENCH | HOW2SCORE: 27.30 -> 37.89 | Improved |
| HOW2SCORE-based RL effect on out-of-domain benchmarks | Qwen3-4B/8B-Instruct, Olmo-3-7B-Think · 12 out-of-domain benchmarks | mean OOD change: +0.52, +0.06, +1.05 | No clear effect |
| add reference-calibrated length reward to RL (ablation) | HOW2SCORE RL · HOW2BENCH | generation length ratio / apparent gain: length ratio up to 1.34x-1.53x without it -> about 1.0 with it | Improved |
| evaluate RL gains with external judges (GPT 5, Gemini 2.5 Pro) instead of distilled judge | RL-trained models · HOW2BENCH | HOW2SCORE | No clear effect |
| RL on Qwen3-8B, blinded manual evaluation | Qwen3-8B · 200 HOW2BENCH examples | no-failure rate (%): 32.5% -> 40.0% | Improved |
| add SFT stage on instruct checkpoints | instruction-tuned checkpoints · HOW2BENCH | HOW2SCORE | No clear effect |
| start post-training from later Olmo 3 7B pretraining checkpoints (RL gain) (ablation) | Olmo 3 7B SFT then RL · HOW2BENCH | HOW2SCORE RL gain: 3.56 at 10% -> 20.33 at 100% | Improved |
| start SFT from later Olmo 3 7B pretraining checkpoints (SFT gain) (ablation) | Olmo 3 7B SFT · HOW2BENCH | HOW2SCORE SFT gain: 3.39 to 5.36 | No clear effect |
| RL on all topics vs science-only vs dining-only (ablation) | Qwen3-8B RL · HOW2BENCH | overall HOW2SCORE gain: all +10.10; science-only +9.41; dining-only +5.55 | Mixed |
| repeat source documents up to 10 times during midtraining (ablation) | Olmo 3 7B · midtraining memorization eval | HOW2SCORE: 14.0 -> 16.5 (document perplexity 10.4 -> 1.4) | No clear effect |
Closest papers by meaning
- ReGenesis: LLMs can Grow into Reasoning Generalists via Self-Improvement (ICLR 2025 · similarity 0.88)
- Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators (ICML 2025 · similarity 0.88)
- JudgeBench: A Benchmark for Evaluating LLM-Based Judges (ICLR 2025 · similarity 0.88)
- EvaLearn: Quantifying the Learning Capability and Efficiency of LLMs via Sequential Problem Solving (NeurIPS 2025 · similarity 0.87)
- StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solving (NeurIPS 2024 · similarity 0.87)
- Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
