Research map / Architectures and efficiency / Attention and transformers
CogLTX: Applying BERT to Long Texts
Ming Ding, Chang Zhou, Hongxia Yang, Jie Tang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multi-step reasoning (ablation) | CogLTX · HotpotQA distractor dev | Joint F1: 65.35 -> 69.21 | Improved |
| add rehearsal and decay (fine scores) (ablation) | CogLTX · HotpotQA distractor dev | Sup F1: 47.37 -> 85.78 | Improved |
| add rehearsal and decay (fine scores) (ablation) | CogLTX · HotpotQA distractor dev | Joint F1: 37.74 -> 69.21 | Improved |
| train reasoner on MemRecall-matched blocks instead of random blocks (ablation) | CogLTX · HotpotQA distractor dev | Joint F1: 66.90 -> 69.21 | Improved |
| add intervention-based label adjustment (ablation) | CogLTX Glove init · 20NewsGroups | accuracy: 84.8 -> 87.0 | Improved |
| Glove instead of Bm25 initialization of relevance labels (ablation) | CogLTX · 20NewsGroups | accuracy: 86.1 -> 87.0 | Improved |
| evaluate only on texts longer than 512 tokens (ablation) | CogLTX · 20NewsGroups | accuracy: 87.0 -> 87.4 | No clear effect |
| CogLTX instead of RoBERTa-large sliding window | CogLTX-base · NewsQA | F1: 66.3 -> 70.1 | Improved |
| CogLTX instead of sliding window with max-pooling | CogLTX · Alibaba | Macro-F1: 89.9 -> 97.2 | Improved |
| CogLTX vs SAE-large (specialized selector) | CogLTX · HotpotQA distractor dev | Joint F1: 71.45 -> 69.21 | Hurt |
Closest papers by meaning
- Longer Context, Deeper Thinking: Uncovering the Role of Long-Context Ability in Reasoning (NeurIPS 2025 · similarity 0.85)
- Chain of Agents: Large Language Models Collaborating on Long-Context Tasks (NeurIPS 2024 · similarity 0.85)
- Enhancing LLM’s Cognition via Structurization (NeurIPS 2024 · similarity 0.84)
- Long-form factuality in large language models (NeurIPS 2024 · similarity 0.84)
- M+: Extending MemoryLLM with Scalable Long-Term Memory (ICML 2025 · similarity 0.84)
- PaceLLM: Brain-Inspired Large Language Models for Long-Context Understanding (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
