Research map / Large language models / Instruction and fine-tuning
Inverse IFEval: Can LLMs Unlearn Stubborn Training Conventions to Follow Real Instructions?
Qinyan Zhang, Xinping Lei, Ruijie Miao, FU YU, Haojie Fan, Le Chang, Jiafan Hou, Dingling Zhang, Zhongfei Hou, ZiqiangYang, puchangxin, FEI HU and 8 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use thinking mode instead of non-thinking mode (ablation) | Qwen3 series · Inverse IFEval (English) | overall score: Qwen3-235B-A22B 40.28 (Instruct) -> 54.22 (Thinking) | Improved |
| use thinking mode instead of non-thinking mode (ablation) | Qwen3-30B-A3B · Inverse IFEval (English) | overall score: 30.43 -> 49.21 | Improved |
| thinking mode vs auto vs non-thinking (ablation) | Doubao-Seed-1.6 (AdaCoT) · Inverse IFEval | overall score | Improved |
| use auto-thinking mode instead of non-thinking (ablation) | Doubao-Seed-1.6 (AdaCoT) · Inverse IFEval (Chinese) | overall score | Hurt |
| use full thinking budget (Pro) instead of reduced budget (Flash) (ablation) | Gemini-2.5 · Inverse IFEval (English) | overall score: 68.61 -> 70.55 | Improved |
| scale up model parameters (ablation) | Qwen3 series · Inverse IFEval | overall score: English Instruct 30.43 -> 40.28 (30B-A3B -> 235B-A22B) | Improved |
| increase Best-of-N from 1 to 16 and 32 (ablation) | five models · Inverse IFEval (avg EN+ZH) | overall score | Improved |
| evaluate counterintuitive instructions instead of conventional IFEval instructions | Qwen3-235B-A22B-Instruct · IFEval vs Inverse IFEval ranking | rank among 15 models: 5th -> 15th | Hurt |
| compare Chinese-developed models in Chinese vs English | GLM-4.5 and Qwen3-235B-A22B-Thinking · Inverse IFEval | overall score: GLM-4.5 58.30 -> 66.96; Qwen3-235B-Thinking 54.22 -> 70.62 | Improved |
| optimize judge model, template and system prompt per instruction type | LLM-as-a-Judge · judge validation | judging accuracy: 88% -> 98% | Improved |
Closest papers by meaning
- The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning (ICLR 2024 · similarity 0.88)
- LLM Alignment--Utility Asymmetry under Semantic-Preserving Transformations (NeurIPS 2026 · similarity 0.88)
- InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instruct (AAAI 2025 · similarity 0.87)
- Delving into the Reversal Curse: How Far Can Large Language Models Generalize? (NeurIPS 2024 · similarity 0.87)
- TAIA: Large Language Models are Out-of-Distribution Data Learners (NeurIPS 2024 · similarity 0.87)
- Remarkable Robustness of LLMs: Stages of Inference? (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
