Research map / Large language models / Language, safety and interpretability
Exploring Large Language Model for Graph Data Understanding in Online Job Recommendations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fine-tunes a LoRA-adapted BELLE-LLaMA-7B recommender on job-resume data with meta-path prompts plus path shuffling and soft path selection to understand behavior graphs for job recommendation.
Problem
LLM recommenders cannot understand non-textual behavior graphs in online recruitment, which limits personalized job recommendation and out-of-distribution (OOD) generalization; path-based prompts also introduce weight and position biases.
Approach
A meta-path prompt constructor converts heterogeneous behavior-graph meta-paths into natural language prompts for an instruction-tuned LLM, with a path augmentation module (shuffle, soft path selector, hybrid) to reduce order and weighting bias.
Key results
- GLRec reaches AUC 0.891 on RecrX point-wise random split vs 0.842 for TALLRec (best baseline).
- On OOD_position split, GLRec achieves 0.810 (RecrX) vs 0.770 for TALLRec.
- AUC for GLRec rises from about 0.71 with no meta-paths to about 0.88 with 2-3 meta-paths on the random split.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add behavior meta-path prompts (ablation) | GLRec · RecrX point-wise Random | AUC: 0.71 to 0.88 | Helped |
| increase the number of meta-paths from 0 to 1 (ablation) | GLRec · RecrX point-wise Random | AUC | Helped |
| increase the number of meta-paths from 1 to 2 or 3 (ablation) | GLRec · RecrX point-wise Random | AUC | Helped |
| add meta-path prompts (ablation) | RobertaRec · RecrX point-wise Random | AUC | No clear effect |
| shuffle meta-path prompt order during training (ablation) | GLRec · RecrX point-wise | AUC | Helped |
| add a learned soft selector for meta-path prompts (ablation) | GLRec · RecrX point-wise | AUC | Helped |
| combine prompt shuffling and the learned soft selector (ablation) | GLRec · RecrX point-wise | AUC | Helped |
Surprising findings
- Pair-wise job matching is easier than point-wise matching for most models.
- RobertaRec's AUC stays roughly flat as meta-path prompts are added, while the LLM benefits substantially.
Limitations
- Evaluated only on two proprietary Chinese recruitment datasets with AUC as the sole metric; no ranking metrics.
- Performance on OOD splits drops relative to random split and is described as near random for some baselines.
- Path selection and sizes (3-node paths, M up to 3) were chosen heuristically based on experimental feedback.
Experimental setup
Datasets: RecrX RecrY
Benchmarks: RecrX RecrY
Metrics: AUC
Baselines: RobertaRec HGT DPGNN TALLRec
Builds on: LoRA TALLRec HGT meta-path
Closest papers by meaning
- Can Large Language Models Analyze Graphs like Professionals? A Benchmark, Datasets and Models (NeurIPS 2024 · similarity 0.88)
- Language Representations Can be What Recommenders Need: Findings and Potentials (ICLR 2025 · similarity 0.87)
- GraphRouter: A Graph-based Router for LLM Selections (ICLR 2025 · similarity 0.87)
- LLMRG: Improving Recommendations through Large Language Model Reasoning Graphs (AAAI 2024 · similarity 0.86)
- Improving LLMs for Recommendation with Out-Of-Vocabulary Tokens (ICML 2025 · similarity 0.86)
- Bridging the User-side Knowledge Gap in Knowledge-aware Recommendations with Large Language Models (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.