AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

CGMIS: Concept-Graph Based Multi-Hop Instructions Synthesis for Enhancing Long-Context Reasoning

Zechen Sun, Zecheng Tang, Juntao Li, Wenpeng Hu, Wenliang Chen, Zhunchen Luo, Qiaoming Zhu

AAAI 2026 · Large language models › Reasoning and chain of thought · approach: Search and self-correction in reasoning · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CGMIS, a concept-graph-based pipeline that synthesizes multi-hop instruction data with explicit intermediate reasoning steps to improve LLM long-context reasoning.

Problem

Scarcity of high-quality multi-hop instruction data for long-context reasoning, and existing synthesis methods do not explicitly model intermediate reasoning steps, making the data unverifiable.

Approach

Builds a concept graph to structure multi-hop instruction synthesis so that intermediate reasoning steps are explicitly modeled and the resulting instructions are verifiable.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
fine-tune on 1K CGMIS multi-hop dataLLaMA-3-8B-Instruct · LongBench avgaverage score: 43.7 -> 48.1Improved
fine-tune on 1K CGMIS multi-hop dataLLaMA-3.1-8B-Instruct · LongBench avgaverage score: 52.0 -> 53.1Improved
fine-tune on 1K CGMIS multi-hop dataQwen-2.5-14B-Instruct · LongBench avgaverage score: 48.0 -> 53.1Improved
fine-tune on LongAlign (10K)LLaMA-3.1-8B-Instruct · LongBench avgaverage score: 52.0 -> 45.9Hurt
fine-tune on LongMIT-en (18K)Qwen-2.5-7B-Instruct · LongBench avgaverage score: 39.8 -> 33.9Hurt
fine-tune on CGMISQwen-2.5-14B-Instruct · LongBench Codeaverage score: 35.3 -> 53.3Improved
fine-tune on CGMISLLaMA-3 models · LongBench Multi-Document QAaverage score: LLaMA-3: HQA 48.2 -> 52.3 but MuSiQue-type tasks vary; slightly underperforms vs LongAlpacaMixed
regenerate 4-hop data with CGMIS framework instead of LongMIT from same documents (ablation)LLaMA-3.1-8B-Instruct · LongBench avgaverage score: 46.4 -> 50.8Improved
regenerate 4-hop data with CGMIS framework instead of LongMIT from same documents (ablation)Qwen-2.5-7B-Instruct · LongBench avgaverage score: 39.4 -> 40.6Improved
increase hops of CGMIS training data from 1 to 4 (ablation)LLaMA-3.1-8B-Instruct · LongBench avgaverage score: 50.2 -> 53.1Improved
increase hops of LongMIT training data from 1 to 4 (ablation)LLaMA-3.1-8B-Instruct · LongBench avgaverage score: 45.4 -> 46.4Improved
scale CGMIS data from 1,000 to 2,000 samples (ablation)LLaMA-3.1-8B-Instruct · LongBench avgaverage score: 53.1 -> 51.7Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy