AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

LiteLong: Resource-Efficient Long-Context Data Synthesis for LLMs

Junlong Jia, Xing Wu, Chaochen Gao, Ziyang Chen, Zijia Lin, Zhongzhi Li, Weinong Wang, Haotian Xu, Donghui Jin, Debing Zhang, Binghui Guo

AAAI 2026 · Large language models › Language, safety and interpretability · approach: LLM pruning and scaling data · 3 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use BISAC categories instead of GPT-4o generated category system (ablation)LiteLong · HELMET + RULERaverage score: 59.94 -> 61.90Improved
add multi-agent debate for topic generation (ablation)LiteLong · HELMET + RULER at 128Kaverage score: 56.33 -> 58.59Improved
increase topic count beyond 0.5x (ablation)LiteLong · HELMET + RULER, 2B tokensaverage score: 49.01 (0.5x) -> 48.56 (1x)No clear effect
reduce topic count to 0.0625x (ablation)LiteLong · HELMET + RULER, 2B tokensaverage score: 49.01 -> 44.80Hurt
mix FineWeb-Edu with Cosmopedia V2 instead of Cosmopedia V2 only (ablation)LiteLong · HELMET + RULERaverage score: 52.32 -> 61.90Improved
mix Cosmopedia V2 into FineWeb-Edu (ablation)LiteLong · HELMET + RULERaverage score: 61.29 -> 61.90Improved
Filter-Reject retention instead of Keep-Accept (ablation)multi-agent debate · HELMET + RULERaverage score: 61.34 -> 61.90Improved
Filter-Reject retention instead of Keep-Fixed-K-Accept (ablation)multi-agent debate · HELMET + RULERaverage score: 61.54 -> 61.90Improved
add NExtLong on top of LiteLongLiteLong · HELMET + RULER averageaverage score: 61.90 -> 63.04Improved
add LiteLong to NExtLongNExtLong · Embedding GPU hoursGPU hours: 928 -> 176 totalImproved
train with LiteLong long-context dataLLaMA-3-8B · Short-context tasks averageaverage score: 62.05 -> 61.92No clear effect
add LiteLong to NExtLong data before SFTNExtLong · LongBench v2overall: 30.4 -> 30.6No clear effect

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy