Research map / Architectures and efficiency / Attention and transformers
COMI: Coarse-to-fine Context Compression via Marginal Information Gain
Jiwei Tang, Shilei Liu, Zhicheng Zhang, Yujin Yuan, Libin Zheng, Wenbo Su, Bo Zheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace uniform grouping with MIG-based group reallocation (ablation) | COMI · NaturalQuestions 32x Qwen2-7B | EM: 54.54 -> 56.31 | Improved |
| replace average pooling with MIG-weighted token merging (ablation) | COMI · NaturalQuestions 32x Qwen2-7B | EM: 50.81 -> 56.31 | Improved |
| add inter-group redundancy to relevance in reallocation (ablation) | COMI · NaturalQuestions 32x Qwen2-7B | EM: 54.59 -> 56.31 | Improved |
| add intra-group redundancy penalty in token merging (ablation) | COMI · NaturalQuestions 32x Qwen2-7B | EM: 51.53 -> 56.31 | Improved |
| add MIG reallocation on 2WikiMQA (ablation) | COMI · 2WikiMQA 32x Qwen2-7B | EM: 49.82 -> 52.13 | Improved |
| use MIG instead of relevance for answer-segment prediction (ablation) | MIG · NaturalQuestions | AUC: 0.5423 -> 0.5809 | Improved |
| use MIG instead of relevance at 25% retention (ablation) | MIG · NaturalQuestions | redundancy score (lower better): 0.6839 -> 0.5673 | Improved |
| COMI vs best baseline at 32x on NaturalQuestions | COMI · NaturalQuestions 32x Qwen2-7B | EM: 24.63 -> 49.15 | Improved |
| COMI vs original prompt with native long-context model | COMI · NaturalQuestions 16x Qwen3-4B | F1: 16.90 -> 34.79 | Improved |
| COMI vs original prompt on strong Qwen2 model at 32x | COMI · NaturalQuestions Qwen2-7B | EM: 72.35 -> 49.15 | Hurt |
| COMI vs Activation Beacon on latency | COMI · NarrativeQA 32x | end-to-end latency (s): 3.93 -> 3.27 | Mixed |
Closest papers by meaning
- Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM Inference (AAAI 2025 · similarity 0.90)
- Compressing LLMs: The Truth is Rarely Pure and Never Simple (ICLR 2024 · similarity 0.89)
- Fundamental Limits of Prompt Compression: A Rate-Distortion Framework for Black-Box Language Models (NeurIPS 2024 · similarity 0.88)
- Leveraging Attention to Effectively Compress Prompts for Long-Context LLMs (AAAI 2025 · similarity 0.88)
- RECOMP: Improving Retrieval-Augmented LMs with Context Compression and Selective Augmentation (ICLR 2024 · similarity 0.88)
- AdmTree: Compressing Lengthy Context with Adaptive Semantic Trees (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
