Research map / Label-efficient and robust learning / Recognition and long-tail
ActionPiece: Contextually Tokenizing Action Sequences for Generative Recommendation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ActionPiece, a context-aware action sequence tokenizer for generative recommendation that treats each action as a feature set, builds a BPE-style vocabulary over within-set and adjacent-set co-occurrences, and uses set permutation regularization for augmentation and ensembling.
Problem
Existing generative recommendation models tokenize each action independently, so identical actions get identical tokens regardless of surrounding context, which limits performance.
Approach
Represent each action as an unordered set of item features and learn a BPE-like vocabulary by merging feature pairs co-occurring within a set or across adjacent sets with probability-based weights; randomly permute features within sets during segmentation (set permutation regularization) to yield multiple segmentations for training augmentation and inference ensembling.
Key results
- ActionPiece achieves NDCG@10 of 0.0264 on Sports, 0.0424 on Beauty, and 0.0451 on CDs, outperforming the strongest baseline by 6.00% to 12.82%.
- Naive segmentation uses only 56.89% of tokens during training, while set permutation regularization reaches 87.01% token utilization after the first epoch.
- Removing set permutation regularization from training or inference degrades NDCG@10 (e.g., Sports 0.0244 training-only, 0.0192 inference-only vs 0.0264 full).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use context-aware cross-action token merging instead of within-action-only merging (ablation) | ActionPiece vocabulary construction · Sports, Beauty, CDs | NDCG@10: 0.0258 -> 0.0264; 0.0416 -> 0.0424; 0.0429 -> 0.0451 | Helped |
| Use weighted rather than equal token-pair counting (ablation) | ActionPiece vocabulary construction · Sports, Beauty, CDs | NDCG@10: 0.0257 -> 0.0264; 0.0412 -> 0.0424; 0.0435 -> 0.0451 | Helped |
| Add vocabulary construction rather than using raw item features as tokens (ablation) | ActionPiece · Sports, Beauty, CDs | NDCG@10: 0.0215 -> 0.0264; 0.0389 -> 0.0424; 0.0346 -> 0.0451 | Helped |
| Use SPR at both training and inference rather than naive segmentation at inference (ablation) | ActionPiece · Sports, Beauty, CDs | NDCG@10: 0.0192 -> 0.0264; 0.0316 -> 0.0424; 0.0329 -> 0.0451 | Helped |
| Use SPR at both training and inference rather than naive segmentation during training (ablation) | ActionPiece · Sports, Beauty, CDs | NDCG@10: 0.0244 -> 0.0264; 0.0387 -> 0.0424; 0.0422 -> 0.0451 | Helped |
| Apply SPR to context-independent TIGER tokenization (ablation) | TIGER · Sports, Beauty, CDs | NDCG@10: TIGER + SPR: 0.0202; 0.0330; 0.0351 | Mixed |
| Vary TIGER vocabulary size to 192, 768, 1k, 49k, or 66k (ablation) | TIGER · Sports, Beauty, CDs | NDCG@10: TIGER variants: 0.0231, 0.0220, 0.0225, 0.0162, 0.0194 (Sports); 0.0362, 0.0378, 0.0384, 0.0317 (Beauty); 0.0331, 0.0411 | Hurt |
| Increase ActionPiece vocabulary size | ActionPiece · Sports, Beauty, CDs | NDCG@10 | Helped |
| Use SPR to segment training sequences | ActionPiece · Sports | token utilization rate: 56.89% -> 87.01% | Helped |
| Use SPR to segment training sequences over additional epochs | ActionPiece · training | token utilization rate: 56.89% to 95.33% by the 5th epoch | Helped |
| Increase the number of inference-time segments | ActionPiece · Sports, Beauty, CDs | NDCG@10 | Helped |
| Use inference-time ensembling with more segments | ActionPiece · higher q, especially q=5 and q=7 | NDCG@10 | Mixed |
Surprising findings
- Applying set permutation regularization to TIGER alone does not help much (Sports NDCG@10 0.0202 vs TIGER 0.0225), indicating SPR needs context-aware tokens to be effective.
- Scaling TIGER's vocabulary up or down does not reproduce ActionPiece's gains, with some TIGER variants underperforming the original.
Limitations
- Inference-time ensembling with q segmentations increases inference FLOPs and cost, with diminishing gains beyond q=5.
- Evaluated only on three Amazon Reviews categories with features quantized via OPQ, with the paper stating that CDs lacks prior GR results for comparison.
Experimental setup
Datasets: Amazon Reviews Sports and Outdoors Amazon Reviews Beauty Amazon Reviews CDs and Vinyl
Benchmarks: Amazon Reviews Sports Amazon Reviews Beauty Amazon Reviews CDs
Metrics: Recall@K NDCG@K
Baselines: BERT4Rec SASRec FDSA S3-Rec VQ-Rec P5-CID TIGER LMIndexer HSTU SPM-SID
Builds on: BPE TIGER SentencePiece T5 OPQ
Closest papers by meaning
- Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations (ICML 2024 · similarity 0.86)
- Action-On-Item Preference Flow: A Shared Event Schema for Predictive and Generative Personalization (NeurIPS 2026 · similarity 0.84)
- Language Model Beats Diffusion - Tokenizer is key to visual generation (ICLR 2024 · similarity 0.84)
- On Discriminative vs. Generative classifiers: Rethinking MLLMs for Action Understanding (ICLR 2026 · similarity 0.84)
- Rethinking Semantic ID Construction for Generative Recommendation: SimHash with Parallel Decoding and Semantic Alignment (NeurIPS 2026 · similarity 0.84)
- Fine Tuning Out-of-Vocabulary Item Recommendation with User Sequence Imagination (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.