Research map / Graph learning / Graph self-supervised learning
Structurally Human, Semantically Biased: Detecting LLM-Generated References with Embeddings and GNNs
Melika Mobini, Vincent Holst, Floriano Tori, Andres Algaba, Vincent Ginis
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use title embeddings instead of graph-property features | Random Forest · Ground truth vs GPT | accuracy: 0.6079 -> 0.8346 | Improved |
| use title embeddings instead of graph-property features | GCN · Ground truth vs GPT | test accuracy: 57.73 -> 93.10 | Improved |
| use title embeddings instead of graph-property features | GIN · Ground truth vs GPT | test accuracy: 51.71 -> 93.28 | Improved |
| use title embeddings instead of graph-property features | GAT · Ground truth vs GPT | test accuracy: 57.40 -> 93.78 | Improved |
| use title embeddings instead of graph-property features | GraphSage · Ground truth vs GPT | test accuracy: 55.59 -> 93.14 | Improved |
| use title embeddings instead of graph-property features | Random Forest · Ground truth vs Claude | accuracy: 0.5389 -> 0.7688 | Improved |
| replace OpenAI embeddings with SPECTER (ablation) | Random Forest · Ground truth vs GPT | accuracy: 0.8346 -> 0.7502 | Hurt |
| replace OpenAI embeddings with SPECTER (ablation) | Random Forest · Ground truth vs Claude | accuracy: 0.7688 -> 0.6878 | Hurt |
| replace semantic embeddings with i.i.d. random vectors of same dimension (ablation) | Random Forest · Ground truth vs GPT | accuracy: 0.8346 -> 0.5047 | Hurt |
| use temporally constrained random baseline (ablation) | Random Forest on graph properties · Ground truth vs GPT | accuracy: 0.6079 -> 0.6119 | No clear effect |
| swap generator at test time (train GPT-4o, test Claude) | Random Forest on OpenAI embeddings · Ground truth vs generated | accuracy: 0.8346 -> 0.7205 | Hurt |
| swap generator at test time (train GPT-4o, test Claude) | GCN · Ground truth vs generated | accuracy: 93.10 -> 0.8001 | Hurt |
Closest papers by meaning
- HLM-Cite: Hybrid Language Model Workflow for Text-based Scientific Citation Prediction (NeurIPS 2024 · similarity 0.88)
- Graph-KV: Breaking Sequence via Injecting Structural Biases into Large Language Models (NeurIPS 2025 · similarity 0.87)
- Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning (ICLR 2024 · similarity 0.86)
- CiteME: Can Language Models Accurately Cite Scientific Claims? (NeurIPS 2024 · similarity 0.85)
- TEG-DB: A Comprehensive Dataset and Benchmark of Textual-Edge Graphs (NeurIPS 2024 · similarity 0.85)
- Generalization Principles for Inference over Text-Attributed Graphs with Large Language Models (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
