Research map / Graph learning / Graph self-supervised learning
Towards Improved Sentence Representations using Token Graphs
Krishna Sri Ipsit Mantri, Carola-Bibiane Schönlieb, Zorah Lähner, Moshe Eliasof
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace mean/max/AdaPool pooling with GLOT token-graph pooling | frozen BERT · GLUE CoLA | MCC: 29.20 -> 47.49 | Improved |
| replace pooling with GLOT | frozen backbones · GLUE (10 tasks, 6 backbones) | mixed metrics: e.g. MRPC BERT 81.64 -> 82.58 | Mixed |
| replace pooling with GLOT | frozen backbones · IMDB | accuracy: 90.91 -> 94.52 (RoBERTa, vs AdaPool) | Improved |
| replace pooling with GLOT | frozen RoBERTa · MTEB SciFact | NDCG@10: 0.0825 -> 0.2605 (vs Mean) | Improved |
| replace pooling with GLOT | frozen SmolLM2 · MTEB SciFact | NDCG@10: 0.1702 -> 0.1834 (vs AdaPool) | No clear effect |
| add token-graph relational learning vs independent-token pooling under 90% distractors | GLOT · diagnostic stress test | accuracy: 78.4 -> over 97 (vs AdaPool, Mistral-7B) | Improved |
| prune graph edges with similarity threshold tau instead of fully connected graph (tau 0.0 to 0.4/0.6) (ablation) | GLOT · GLUE QNLI / CoLA (Mistral-7B) | accuracy / MCC: QNLI 52.85 -> 60.93; CoLA 50.19 -> 54.30 | Mixed |
| use GLOT with frozen backbone instead of LoRA/full fine-tuning | Mistral-7B · CoLA | MCC: 48.23 -> 53.29 (vs LoRA) | Improved |
Closest papers by meaning
- LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddings (NeurIPS 2024 · similarity 0.87)
- Large Language Model Meets Graph Neural Network in Knowledge Distillation (AAAI 2025 · similarity 0.87)
- Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning (ICLR 2024 · similarity 0.86)
- A Unified Graph Language Model for Multi-Domain Multi-Task Graph Alignment Instruction Tuning (NeurIPS 2026 · similarity 0.86)
- What Kind of Visual Tokens Do We Need? Training-Free Visual Token Pruning for Multi-Modal Large Language Models from the Perspective of Graph (AAAI 2025 · similarity 0.86)
- GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
