AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

Kalman Filtering Attention for User Behavior Modeling in CTR Prediction

Hu Liu, Jing LU, Xiwei Zhao, Sulong Xu, Hao Peng, Yutong Liu, Zehua Zhang, Jian Li, Junsheng Jin, Yongjun Bao, Weipeng Yan

NeurIPS 2020 · Architectures and efficiency › Attention and transformers · approach: Theory of self-attention dynamics · 9 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Replace Transformer attention with KFAtt-freqTransformer behavior model · Amazon Electronics (All)AUC: 0.8720 -> 0.8789Improved
Replace Transformer attention with KFAtt-freqTransformer behavior model · Amazon Electronics (New queries)AUC: 0.8488 -> 0.8578Improved
Plug KFAtt-freq into vanilla attentionVanilla Attention · Amazon Electronics (All)AUC: 0.8034 -> 0.8481Improved
Plug KFAtt-freq into DINDIN · Amazon Electronics (All)AUC: 0.8317 -> 0.8524Improved
Add global query prior (KFAtt-base) to TransformerTransformer · Amazon Electronics (All)AUC: 0.8720 -> 0.8766Improved
Add frequency capping on top of KFAtt-base (KFAtt-freq)KFAtt-trans · Amazon Electronics (All)AUC: 0.8766 -> 0.8789Improved
Learned query-specific prior (sigma_q) instead of naive fixed prior sigma_q=1 (ablation)KFAtt-base · Amazon Electronics (All)AUC: 0.8740 -> 0.8766Improved
Include random error term in frequency capping rather than naive capping (ablation)KFAtt-freq · Amazon Electronics (All)AUC: 0.8754 -> 0.8789Improved
Coarser (2nd/1st-level category) deduplication instead of 3rd-level (ablation)KFAtt-freq · Amazon Electronics (All)AUC: 0.8789 -> 0.8775 / 0.8766Hurt
Coarser (2nd/1st-level category) deduplication instead of 3rd-level (ablation)KFAtt-freq · Amazon Electronics (Infrequent queries)AUC: 0.8496 -> 0.8504 / 0.8506Improved
Deploy KFAtt-freq online vs sum-pooling base modelJD.com ranking system · JD.com online A/B testAUC: 4.4% CTR gainImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy