Research map / Large language models / Retrieval and knowledge
Counterfactual Question Generation Uncovering Learner Contradictions
Bo Zhang, Hao Yu, Wenjie Dong, Yvhang Yang, Dezhuang Miao, Fengyi Song, Yanhui Gu, Xiaoming Zhang, Junsheng Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add KGMap schema memory to GapProbe (ablation) | GapProbe · CWQ-CFQG test | Cognitive Depth (DeepSeek-V3): 0.748 -> 0.949 | Improved |
| add KGMap schema memory to GapProbe (ablation) | GapProbe · CWQ-CFQG test | Good CFQ rate (%) (DeepSeek-V3): 49.5 -> 62.1 | Improved |
| add KGMap schema memory to GapProbe (ablation) | GapProbe · CWQ-CFQG test | Good CFQ rate (%) (GPT-4): 42.3 -> 47.6 | Improved |
| add KGMap schema memory to GapProbe (ablation) | GapProbe · CWQ-CFQG test | Reflection Trigger (DeepSeek-V3): 0.738 -> 0.672 | Hurt |
| add KGMap schema memory to GapProbe (ablation) | GapProbe · CWQ-CFQG test | Knowledge Focus: 0.937 -> 0.963 (GPT-4); 0.972 -> 0.960 (DS-v3) | No clear effect |
| use ProConB iterative cycle instead of MFC-style single pass | CFQ generation · CWQ-CFQG test | Reflection Trigger (GPT-4): 0.377 -> 0.505 (w/o KGMap) | Improved |
| add confusability filtering (+P) (ablation) | GapProbe · CWQ-CFQG test | RT and Good rate; KF | Mixed |
| add multi-hop SPARQL generation (ablation) | GapProbe · CWQ-CFQG test | Cognitive Depth | Improved |
| use GapProbe-QA instead of MFC | GPT-4 KBQA · KBQA average of four benchmarks | EM accuracy (%): 60.9 -> 66.9 | Improved |
| use GapProbe-QA instead of ToG | GPT-4 KBQA · CWQ-QQA | EM accuracy (%): 58.1 -> 55.5 | Hurt |
| use GapProbe-QA instead of ToG | DeepSeek-V3 KBQA · CWQ | EM accuracy (%): 56.3 -> 58.1 (ToG vs GapProbe-QA); MFC 60.0 | Mixed |
| add CFQ feedback to student LLM reflection | Llama-3-70B · LLM-as-student simulation | correction rate (%): 8.70 -> 33.3 | Improved |
Closest papers by meaning
- Reasoning Elicitation in Language Models via Counterfactual Feedback (ICLR 2025 · similarity 0.87)
- CofCA: A STEP-WISE Counterfactual Multi-hop QA benchmark (ICLR 2025 · similarity 0.87)
- Walk the Talk? Measuring the Faithfulness of Large Language Model Explanations (ICLR 2025 · similarity 0.86)
- Decompose, Analyze and Rethink: Solving Intricate Problems with Human-like Reasoning Cycle (NeurIPS 2024 · similarity 0.85)
- AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions (NeurIPS 2025 · similarity 0.85)
- Debate on Graph: A Flexible and Reliable Reasoning Framework for Large Language Models (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
