Research map / Large language models / Instruction and fine-tuning
REEF: Representation Encoding Fingerprints for Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes REEF, a training-free LLM fingerprinting method that uses linear or RBF centered kernel alignment (CKA) between a suspect and victim model's representations on the same samples to detect whether the suspect derives from the victim.
Problem
Identifying whether a suspect LLM is a subsequent development (fine-tuned, pruned, merged, permuted) of a victim open-source LLM, in a way that is robust to pruning, permutations, extensive fine-tuning, and merging and does not impair model capabilities.
Approach
Compare representations of suspect and victim models on the same inputs using CKA, which is invariant to column permutations and scaling and works across different representation dimensions, rather than training classifiers or comparing weights/logits.
Key results
- Average CKA similarity between victim (Llama-2-7b) and derived models is 0.9585 versus 0.2361 for unrelated LLMs.
- REEF achieves 0.9962 similarity on Llemma-7b (700B fine-tuning tokens) while PCS drops to 0.0098 and ICS to 0.2257.
- CKA similarity is exactly 1.0000 under column permutation and scaling transformations, whereas ICS, PCS, and Logits drop to near 0 under permutation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use REEF representation CKA instead of weight-based PCS | Llama-2-7b fingerprinting under permutation · Llama-2-7b (permutation) | similarity: 0.0000 -> 1.0000 | Helped |
| Use REEF representation CKA instead of ICS | Llama-2-7b fingerprinting under permutation · Llama-2-7b (permutation) | similarity: 0.1918 -> 1.0000 | Helped |
| Use REEF representation CKA instead of Logits | Llama-2-7b fingerprinting under distribution merging · Fusechat-7b: Internlm2-chat-20b | similarity: 0.0000 -> 0.9278 | Helped |
| Use REEF representation CKA instead of PCS | Llama-2-7b fingerprinting under distribution merging · Fusellm-7b: Openllama-2-7b | similarity: 0.0194 -> 0.6713 | Helped |
| Use REEF representation CKA instead of Logits | Llama-2-7b fingerprinting after extensive fine-tuning · Llemma-7b (700B tokens fine-tuning) | similarity: 0.6367 -> 0.9962 | Helped |
| Use a single layer's CKA similarity for fingerprint identification | REEF on Llama-2-7b · TruthfulQA, layer 18 (derived models vs unrelated models) | similarity: 0.9973 and 0.2223 | Mixed |
| Use linear CKA rather than RBF CKA | REEF model fingerprinting · TruthfulQA | similarity: > 0.95 | No clear effect |
| Increase sample count until approximately 200-300 samples (ablation) | REEF · TruthfulQA | similarity stability: 200-300 | Helped |
| Use TruthfulQA rather than ToxiGen (ablation) | REEF · TruthfulQA and ToxiGen | similarity gap: approximately 0.8 on TruthfulQA and about 0.5 on ToxiGen | Helped |
Surprising findings
- Logits and PCS fail on distribution-merged models with different architectures, while REEF is the only method still identifying victims in Fusechat-7b.
Limitations
- Evaluated only on Llama-2-based victims in most experiments.
- Fingerprint quality for unrelated models with the same architecture depends on dataset choice (TruthfulQA gap largest).
Experimental setup
Datasets: TruthfulQA SST2 ConfAIde PKU-SafeRLHF ToxiGen
Benchmarks: TruthfulQA SST2 ConfAIde PKU-SafeRLHF ToxiGen
Metrics: CKA similarity classification accuracy perplexity
Baselines: PCS ICS Logits
Builds on: centered kernel alignment Hilbert-Schmidt Independence Criterion ICS LLM-Pruner
Closest papers by meaning
- AttnDiff: Attention-based Differential Fingerprinting for Large Language Models (NeurIPS 2026 · similarity 0.86)
- HuRef: HUman-REadable Fingerprint for Large Language Models (NeurIPS 2024 · similarity 0.85)
- Differentially Private Steering for Large Language Model Alignment (ICLR 2025 · similarity 0.85)
- STAMP Your Content: Proving Dataset Membership via Watermarked Rephrasings (ICML 2025 · similarity 0.84)
- On Large Language Model Continual Unlearning (ICLR 2025 · similarity 0.84)
- Dictionary-Aligned Concept Control for Safeguarding Multimodal LLMs (CVPR 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.