Research map / Large language models / Instruction and fine-tuning
Automatically Generating Numerous Context-Driven SFT Data for LLMs Across Diverse Granularity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AugCon, which generates multi-granularity context-driven SFT query-response pairs via a recursive Context-Split-Tree, contrastive-trained scorer filtering, and principle-driven self-alignment with self-improvement for responses.
Problem
Automatically constructing diverse, high-quality, and faithful query-response SFT data from custom corpora at multiple contextual granularities, since human annotation is costly and prior automated methods produce homogeneous queries.
Approach
Recursively derive a query and split each context into two independent sub-contexts (Context-Split-Tree) to cover all granularities; train a contrastive scorer on positive queries versus prompt-degraded negatives to rank queries; filter with ROUGE-L diversity thresholds; generate responses via principle-driven self-alignment with random-search ICL example selection and self-evaluation.
Key results
- On automatic benchmarks with Llama3-70B-Instruct, AugCon achieves the best scores on all four datasets, e.g., SQuAD1.1 accuracy 0.336 vs 0.314 for Context-Instruct (second best).
- Human evaluation on the DailyM test set shows AugCon surpasses baselines on query realism, diversity, relevance, accuracy and satisfaction.
- On WebGLM-QA, AugCon reaches BERTScore 0.924 versus 0.903 for ETRC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AugCon instead of Context-Instruct to generate SFT data | Llama3-70B-Instruct fine-tuned model · SQuAD1.1 | accuracy: 0.314 -> 0.336 | Helped |
| Use AugCon instead of Context-Instruct to generate SFT data | Llama3-70B-Instruct fine-tuned model · TriviaQA | accuracy: 0.825 -> 0.849 | Helped |
| Use AugCon instead of Context-Instruct to generate SFT data | Llama3-70B-Instruct fine-tuned model · DROP | accuracy: 0.334 -> 0.350 | Helped |
| Use AugCon instead of Context-Instruct to generate SFT data | Llama3-70B-Instruct fine-tuned model · WebGLM-QA | BERTScore: 0.885 -> 0.924 | Helped |
Surprising findings
- Fine-tuning Llama3-70B-Instruct on the Llama3-70B-Instruct baseline's own-generated data (Chat Model) performs worse than DAPT and AdaptLLM on short-form QA in the reported table.
Limitations
- Generalization across languages and domains is untested beyond the specific Chinese and English settings.
- Depends heavily on context quality and depth; narrow contexts may yield less diverse data.
- Evaluation metrics may not fully capture semantic fidelity and context appropriateness.
Experimental setup
Datasets: DailyM corpus (1,000 Chinese magazine articles) SQuAD1.1 TriviaQA DROP WebGLM-QA
Benchmarks: DailyM test set SQuAD1.1 TriviaQA DROP WebGLM-QA
Metrics: realism diversity relevance accuracy satisfaction accuracy (short-form) BERTScore
Baselines: Chat Model DAPT AdaptLLM ETRC Context-Instruct
Builds on: AdaptLLM ETRC Context-Instruct DAPT Self-Align
Closest papers by meaning
- Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models (ICML 2024 · similarity 0.87)
- RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs (NeurIPS 2024 · similarity 0.87)
- Preference-Oriented Supervised Fine-Tuning: Favoring Target Model over Aligned Large Language Models (AAAI 2025 · similarity 0.86)
- Skrull: Towards Efficient Long Context Fine-tuning through Dynamic Data Scheduling (NeurIPS 2025 · similarity 0.86)
- Supervised Knowledge Makes Large Language Models Better In-context Learners (ICLR 2024 · similarity 0.86)
- GraphRouter: A Graph-based Router for LLM Selections (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.