Research map / Large language models / Retrieval and knowledge
Diagnosing and Remedying Knowledge Deficiencies in LLMs via Label-free Curricular Meaningful Learning
Kai Xiong, Xiao Ding, Yixin Cao, Li Du, Jiahao Ying, Yang Zhao, Bing Qin, Ting Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add curricular ordering of synthesized data (ablation) | LaMer on Mistral-7B · Seven-benchmark average | accuracy: 55.64 -> 56.09 | Improved |
| add curricular ordering of synthesized data (ablation) | LaMer on LLaMA-3-8B · Seven-benchmark average | accuracy: 64.47 -> 64.70 | Improved |
| add curricular ordering of synthesized data (ablation) | LaMer on Qwen2-7B · Seven-benchmark average | accuracy: 65.50 -> 66.13 | Improved |
| add curricular ordering of synthesized data (ablation) | LaMer on Gemma-1.1-2B · Seven-benchmark average | accuracy: 33.80 -> 34.08 | Improved |
| use relative entropy instead of perplexity for diagnosis | deficiency diagnosis · e-CARE (Mistral-7B-Instruct-v0.2) | F1: 40.10 -> 49.58 | Improved |
| use relative entropy instead of random selection | deficiency diagnosis · e-CARE (Mistral-7B-Instruct-v0.2) | F1: 35.23 -> 49.58 | Improved |
| use LaMer instead of Naive augmentation | Mistral-7B · Seven-benchmark average | accuracy: 52.52 -> 56.09 | Improved |
| apply Naive augmentation | LLaMA-3-8B · Seven-benchmark average | accuracy: 62.82 -> 61.90 | Hurt |
| apply Single augmentation | Gemma-1.1-2B · Seven-benchmark average | accuracy: 33.07 -> 31.51 | Hurt |
| apply AugGPT augmentation | LLaMA-3-8B · GSM-Plus | accuracy: 61.93 -> 14.92 | Hurt |
| apply AugGPT augmentation | Mistral-7B · GSM-Plus | accuracy: 29.91 -> 13.78 | Hurt |
| use LaMer instead of label-reliant LLM2LLM | Mistral-7B · Seven-benchmark average | accuracy: 54.00 -> 56.09 | Improved |
Closest papers by meaning
- Towards Effective Evaluations and Comparisons for LLM Unlearning Methods (ICLR 2025 · similarity 0.89)
- Meaningful Learning: Enhancing Abstract Reasoning in Large Language Models via Generic Fact Guidance (NeurIPS 2024 · similarity 0.88)
- Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs (ICLR 2025 · similarity 0.88)
- OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics (NeurIPS 2025 · similarity 0.88)
- KoLA: Carefully Benchmarking World Knowledge of Large Language Models (ICLR 2024 · similarity 0.88)
- Delving into the Reversal Curse: How Far Can Large Language Models Generalize? (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
