Research map / Large language models / Language, safety and interpretability
RHYTHM: Reasoning with Hierarchical Temporal Tokenization for Human Mobility
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RHYTHM, a frozen-LLM framework for human mobility prediction that tokenizes trajectories into daily segments with hierarchical (intra/inter-segment) attention and adds precomputed LLM prompt embeddings to tokens.
Problem
Predicting future human locations from long trajectories is hard due to long-range dependencies and multi-scale daily/weekly periodicity, and existing LSTM, transformer, and LLM methods either ignore periodicity or are costly to adapt.
Approach
Partition each trajectory into daily segments, encode each with intra-segment attention and pooling into one token, apply inter-segment attention, add precomputed LLM prompt embeddings (trajectory info and task description) to tokens, and feed the sequence into a frozen pretrained LLM with an output projection over locations.
Key results
- RHYTHM-Llama-1B reaches Acc@1 of 0.2929 on Kumamoto, versus 0.2864 for the best baseline COLA, a gain of about 2.4% stated in the paper.
- RHYTHM reports a 24.6% reduction in training time versus the best-performing baseline on Sapporo, and a 5.0% gain on weekends in the daily/weekly analysis.
- Ablation: removing temporal tokenization drops Acc@1 on Kumamoto from 0.2929 to 0.2801 (about 5.39% relative drop as stated); removing hierarchical attention costs about 0.90%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RHYTHM rather than the strongest baseline | Human mobility prediction · Kumamoto, Sapporo, and Hiroshima | Accuracy@1: 2.4% | Helped |
| Add temporal tokenization (ablation) | RHYTHM · Three mobility datasets | prediction performance: 5.39% | Helped |
| Add hierarchical attention (ablation) | RHYTHM · Three mobility datasets | prediction performance: 0.90% | Helped |
| Add trajectory information prompts (ablation) | RHYTHM · Three mobility datasets | prediction performance: 0.10% | Helped |
| Add task description prompts (ablation) | RHYTHM · Three mobility datasets | prediction performance: 0.10% | Helped |
| Use a daily segment length of 48 rather than 1, 24, or 96 (ablation) | RHYTHM · Sapporo | Accuracy@1: 0.2801; 0.2883; 0.2929; 0.2851 | Helped |
| Use a pretrained LLM instead of a randomly initialized frozen LLM (ablation) | RHYTHM · Kumamoto | Accuracy@1: 0.2556 -> 0.2929 | Helped |
| Use a pretrained LLM rather than no LLM (ablation) | RHYTHM · Kumamoto | Accuracy@1: 0.2749 -> 0.2929 | Helped |
| Use non-autoregressive rather than autoregressive prediction (ablation) | RHYTHM · Sapporo | Accuracy@1: 0.2884 -> 0.2929 | Helped |
| Increase backbone from Llama-3.2-1B to Llama-3.2-3B (ablation) | RHYTHM · Sapporo | Accuracy@1: 0.2929 -> 0.2941 | Helped |
Surprising findings
- COLA achieves the best BLEU on all cities, which the authors attribute to its post-hoc recalibration of location frequencies.
- A randomly initialized frozen LLM performs much worse (Acc@1 0.2556) than the pretrained one, while the no-LLM attention-only variant gets 0.2749.
Limitations
- Performance depends on the quality of pretrained LLMs, which are designed for language rather than mobility.
- Non-autoregressive holistic prediction is used rather than step-by-step decoding; training time remains high.
Experimental setup
Datasets: YJMob100K (Kumamoto, Sapporo, Hiroshima)
Benchmarks: YJMob100K Kumamoto YJMob100K Sapporo YJMob100K Hiroshima
Metrics: Accuracy@1 Accuracy@3 Accuracy@5 MRR DTW BLEU
Baselines: LSTM DeepMove PatchTST iTransformer Time-LLM CMHSA PMT COLA ST-MoE-BERT Mobility-LLM
Builds on: Time-LLM FPT PatchTST iTransformer
Closest papers by meaning
- TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond (NeurIPS 2026 · similarity 0.85)
- Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory Prediction (CVPR 2024 · similarity 0.85)
- Large Language Models as Urban Residents: An LLM Agent Framework for Personal Mobility Generation (NeurIPS 2024 · similarity 0.85)
- Spatial-Temporal Interplay in Human Mobility: A Hierarchical Reinforcement Learning Approach with Hypergraph Representation (AAAI 2024 · similarity 0.85)
- AutoTraces: Autoregressive Trajectory Forecasting via Multimodal Large Language Models (CVPR 2026 · similarity 0.85)
- Holistic Semantic Representation for Navigational Trajectory Generation (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.