Research map / Graph learning / Graph self-supervised learning
FoGE: Fock Space inspired encoding for graph prompting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FoGE, a parameter-free graph encoder based on Fock-space/VSA-style operations (circular convolution binding, vector addition superposition) and uses a single linear adapter to prefix-tune a frozen Llama2-7B for graph question answering across simple graphs, hypergraphs, and proteins.
Problem
Encoding diverse graph structures (simple graphs, hypergraphs, proteins) into embeddings that a frozen LLM can use for graph question answering, without task-specific or heavy trainable graph encoders.
Approach
A parameter-free, nearly lossless graph embedding built from random high-dimensional vectors for nodes, combining edges via circular convolution (product) and summing them (superposition), with only a linear adapter trained on top of a frozen LLM as a prefix.
Key results
- FoGE-LLM reaches 97.2% on number of nodes and 97.9% on cycle existence in GraphQA, comparable to or better than GraphToken on most tasks.
- FoGE achieves micro F1 99.2 on PPI among unsupervised methods, reported as state-of-the-art, above supervised GAT (97.3) but below GCNII (99.5).
- On HyperGraphQA, FoGE-LLM reaches 95.4% on number of edges versus 27.0% for few-shot prompting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FoGE-LLM instead of GraphToken | Llama2 (7B) graph prompting · GraphQA: number of edges | accuracy: 42.6% -> 45.1% | Helped |
| Use FoGE-LLM instead of GraphToken | Llama2 (7B) graph prompting · GraphQA: cycle existence | accuracy: 95.6% -> 97.9% | Helped |
| Use FoGE-LLM instead of GraphToken | Llama2 (7B) graph prompting · GraphQA: number of nodes | accuracy: 99.6% -> 97.2% | Hurt |
| Use FoGE-LLM instead of GraphToken | Llama2 (7B) graph prompting · GraphQA: edge existence | accuracy: 73.8% -> 74.3% | Helped |
| Add molecular fingerprint features to FoGE (ablation) | FoGE on mol-HIV · mol-HIV | ROC-AUC: 0.7614±0.0051 -> 0.8305±0.0068 | Helped |
| Use FoGE-LLM instead of few-shot textual graph prompting | Llama2 (7B) graph prompting · HyperGraphQA: number of nodes | accuracy: 16.8% -> 85.0% | Helped |
| Use FoGE-LLM instead of GraphLLM | Llama2 graph prompting · GraphReasoning: substructure count | accuracy: 99.9% -> 97.3% | Hurt |
| Use RoBERTa vectors instead of random vectors (ablation) | FoGE-LLM · GraphReasoning: bipartite match | accuracy: 98.1% -> 97.3% | Hurt |
| Use RoBERTa vectors instead of random vectors (ablation) | FoGE-LLM · GraphReasoning: max triplet sum | accuracy: 94.6% -> 94.7% | Helped |
| Use RoBERTa vectors instead of random vectors (ablation) | FoGE-LLM · GraphReasoning: shortest path | accuracy: 95.7% -> 95.8% | Helped |
| Increase FoGE vector dimensionality (ablation) | FoGE on GraphQA · GraphQA tasks | accuracy | Mixed |
| Use one linear adapter per layer instead of a single input-layer adapter (ablation) | FoGE-LLM · graph prompting | performance | No clear effect |
Surprising findings
- A parameter-free encoder with a single linear adapter matches or beats trainable GNN-based GraphToken and GraphLLM (100M params) on several tasks.
Limitations
- Unsupervised embeddings cannot be fine-tuned if they are insufficient for a specific application.
- Random vector generation is impractical for infinitely many concepts, requiring text encoders such as RoBERTa with trial-and-error configuration.
- Hypergraph encoding relies on modifications because aggregating many circular convolutions is numerically unstable.
Experimental setup
Datasets: GraphQA GraphReasoning HyperGraphQA Jaffe PPI OBNB SabDab mol-HIV
Benchmarks: GraphQA GraphReasoning HyperGraphQA Jaffe mol-HIV PPI OBNB
Metrics: accuracy ROC-AUC micro F1 APOP MSE inference time
Baselines: GraphToken GraphLLM zero-shot prompting few-shot prompting GCN GAT GraphSAGE DGI GRACE Node2Vec GCNII LabelProp
Builds on: GraphToken GraphLLM prefix-tuning vector symbolic architectures HRRFormer
Closest papers by meaning
- G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering (NeurIPS 2024 · similarity 0.87)
- Talk like a Graph: Encoding Graphs for Large Language Models (ICLR 2024 · similarity 0.87)
- LLaGA: Large Language and Graph Assistant (ICML 2024 · similarity 0.86)
- GOFA: A Generative One-For-All Model for Joint Graph Language Modeling (ICLR 2025 · similarity 0.86)
- UniGTE: Unified Graph–Text Encoding for Zero-Shot Generalization across Graph Tasks and Domains (NeurIPS 2025 · similarity 0.86)
- Graph Neural Prompting with Large Language Models (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.