Research map / Large language models / Instruction and fine-tuning
ContextIF: Enhancing Instruction-Following through Context Reward
Yule Zhong, Jiacheng Yao, Guoxiu He
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ContextIF generated context | LLaMA3-8B-Instruct · IFEval | IFEval avg: 77.11 -> 83.35 | Improved |
| add ContextIF generated context | LLaMA3-8B-Instruct · LiveBench | score: 46.70 -> 59.90 | Improved |
| replace SPAR-8B with ContextIF-8B | SPAR-8B · Multi-IF turn 3 | accuracy: 51.32 -> 53.51 | Improved |
| use randomly selected demonstrations as context | LLaMA3-8B-Instruct · IFEval | IFEval avg: 77.11 -> 76.97 | Hurt |
| generate context with GPT-4o instead of with RL-trained generator | ContextIF-8B · IFEval | IFEval avg: 83.35 -> 82.77 | Hurt |
| fine-tune generator with SFT on 400 examples instead of RL | LLaMA3-8B-Instruct LLM-context · IFEval | IFEval avg: 80.97 -> 80.34 | Hurt |
| add format reward (ablation) | ContextIF-8B · IFEval | IFEval avg: 81.13 -> 83.35 | Improved |
| add summary reward (ablation) | ContextIF-8B · IFEval | IFEval avg: 79.22 -> 83.35 | Improved |
| add demonstration question reward (ablation) | ContextIF-8B · IFEval | IFEval avg: 81.15 -> 83.35 | Improved |
| add demonstration answer reward (ablation) | ContextIF-8B · IFEval | IFEval avg: 78.50 -> 83.35 | Improved |
| swap judge LLaMA3-70B for Qwen-2.5-72B (ablation) | ContextIF-8B · IFEval | IFEval avg: 83.35 -> 83.38 | No clear effect |
| add ContextIF | LLaMA3-8B-Instruct · MMLU | accuracy: 68.40 -> 70.10 | Improved |
Closest papers by meaning
- Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Models (ICLR 2025 · similarity 0.91)
- SPaR: Self-Play with Tree-Search Refinement to Improve Instruction-Following in Large Language Models (ICLR 2025 · similarity 0.89)
- BLEUBERI: BLEU is a surprisingly effective reward for instruction following (NeurIPS 2025 · similarity 0.88)
- Incentivizing Reasoning for Advanced Instruction-Following of Large Language Models (NeurIPS 2025 · similarity 0.88)
- In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering (ICML 2024 · similarity 0.87)
- Reward Is Enough: LLMs Are In-Context Reinforcement Learners (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
