Research map / Architectures and efficiency / Attention and transformers

LIME: Link-based User-item Interaction Modeling with Decoupled XOR Attention for Efficient Test Time Scaling

NeurIPS 2026 · Architectures and efficiency › Attention and transformers · approach: Linear attention approximations

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LIME, a recommendation ranking architecture that uses learned link embeddings and an XOR attention mask to decouple user and candidate computation, enabling near-constant candidate-side latency and linear history cost while approaching HSTU accuracy.

Problem

Transformer-based ranking models for recommendation have quadratic cost in user history length and linear cost in candidate count, making large candidate sets and long histories too expensive at inference, while two-tower models lack expressiveness.

Approach

Introduce a small set of globally learned link embeddings as keys so that the query-key attention weights between candidates and links are user-independent and can be precomputed and cached; a XOR attention mask restricts attention to bidirectional link-history interactions, making self-attention linear in history length.

Key results

• On KuaiRand-1K, LIME-XOR reaches 0.7448 AUC vs 0.7444 for the HSTU skyline and 0.7389 for TTSN, with LIME-XOR+Window at 0.7453.
• Industrial online A/B test reports +37.9% VC and +28.6% WT for LIME-XOR-ranked candidates versus the baseline; the paper also reports a 38% source-rate gain in the abstract/introduction.
• Latency table shows LIME-XOR at ~2.5 s vs HSTU Skyline ~3.6 s at 16 candidates and ~5.95 s vs ~23.83 s at 65536 candidates (FlashAttention V2 table).

Surprising findings

• Fixed item centroids achieve the best gAUC in the link/key ablation, so the paper does not establish learned links as uniformly superior.
• Five-seed Taobao-Ad results show DIN has best gAUC and HSTU best AUC, with LIME-XOR only best on log loss.

Limitations

• Relies on low-rank approximation assumption; fragmented user interests may need higher rank.
• QK cache must be periodically rebuilt as item catalog evolves, so best suited for early-stage ranking.
• Full production cache-build cost and miss-fallback latency remain unmeasured.

Experimental setup

Datasets: KuaiRand-1K Taobao-Ad industrial logged data

Benchmarks: KuaiRand-1K Taobao-Ad industrial video completion and watch time tasks

Metrics: AUC NE gAUC log loss latency JS divergence cosine similarity

Baselines: TTSN DIN SASRec HSTU Truncated MHA LREA (Linformer) Linear Transformer Longformer VISTA SIM TWIN

Builds on: HSTU Multi-Head Attention Longformer Linformer FuxiCTR

Closest papers by meaning

• UxSID: Semantic-Aware User Interests Modeling for Ultra-Long Sequence (NeurIPS 2026 · similarity 0.88)
• Long-Sequence Recommendation Models Need Decoupled Embeddings (ICLR 2025 · similarity 0.87)
• Wukong: Towards a Scaling Law for Large-Scale Recommendation (ICML 2024 · similarity 0.86)
• LLM-ESR: Large Language Models Enhancement for Long-tailed Sequential Recommendation (NeurIPS 2024 · similarity 0.86)
• Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations (ICML 2024 · similarity 0.86)
• On the Embedding Collapse when Scaling up Recommendation Models (ICML 2024 · similarity 0.86)