Research map / Architectures and efficiency / Attention and transformers
CLAA: Cross-Layer Attention Aggregation for Accelerating LLM Prefill
Bradley McDanel, Steven Li, Harshit Khaitan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GT-Ref token ranking keeping only 10% of tokens | Llama-3.1-8B-Instruct · LongBench avg | average score: 49.32 -> 49.29 | No clear effect |
| aggregate importance scores across 4 consecutive layers (CLAA) instead of a single layer (FastKV) | FastKV · LongBench avg, 10% keep, Llama-3.1-8B | average score: 46.81 -> 47.13 | Improved |
| CLAA vs FastKV difference at a fixed layer is small | CLAA · LongBench individual datasets | average score: +0.39 | No clear effect |
| use multi-layer aggregation instead of a single layer at the collapse layer (ablation) | FastKV · LongBench six-task avg, Llama-3.2-3B, layer 11, 10% keep | accuracy: 35.05 -> 42.45 | Improved |
| use multi-layer aggregation instead of a single layer at the collapse layer (ablation) | FastKV · LongBench six-task avg, Llama-3.1-8B, layer 12, 10% keep | accuracy: 41.80 -> 52.54 | Improved |
| add multi-layer aggregation (CLAA) to single-layer ranking | FastKV · Needle-in-a-Haystack, Llama-3.1-8B, 40% keep | average retrieval score: 0.906 -> 0.909 | Improved |
| add multi-layer aggregation (CLAA) to single-layer ranking | FastKV · RULER 64K, Llama-3.1-8B, 40% keep | average accuracy: 81.28 -> 82.01 | Improved |
| use a 1B speculator for lookahead (Speculative Prefill) instead of simple attention heuristics | token ranking · LongBench avg, Llama-3.1-8B, 10% keep | average score: 47.13 -> 41.99 | Hurt |
| push FastKV pruning layer later (19 instead of 15) (ablation) | FastKV · LongBench avg, Llama-3.1-8B, 10% keep | average score: 47.20 vs CLAA at layer 15: 47.13; TTFT 2054 ms -> 2489 ms | No clear effect |
| use CLAA instead of semantic compression (LLMLingua, Selective Context) | prompt compression · Qasper, Llama-3.1-8B | score: 0.129 -> 0.470 (vs LLMLingua) | Improved |
| defer KV compression for first 4 layers (m=4), costing extra KV memory | CLAA · KV cache size at 10% keep | KV cache GB: 0.1 GB -> 0.3 GB | Hurt |
| use CLAA at 10% keep rate vs FullKV | Llama-3.1-8B · TTFT 10k tokens | TTFT: from roughly 900ms to 550ms | Improved |
Closest papers by meaning
- MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention (NeurIPS 2024 · similarity 0.90)
- Representation Consistency for Accurate and Coherent LLM Answer Aggregation (NeurIPS 2025 · similarity 0.90)
- OrthoRank: Token Selection via Sink Token Orthogonality for Efficient LLM inference (ICML 2025 · similarity 0.89)
- SparQ Attention: Bandwidth-Efficient LLM Inference (ICML 2024 · similarity 0.89)
- QUEST: Query-Aware Sparsity for Efficient Long-Context LLM Inference (ICML 2024 · similarity 0.89)
- FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
