Research map / Large language models / Instruction and fine-tuning
Scalable Fingerprinting of Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Perinucleus sampling for generating LLM fingerprint responses so that tens of thousands of fingerprints can be embedded into Llama-3.1-8B with little utility loss and good persistence after SFT.
Problem
Existing LLM fingerprinting schemes can embed only a few hundred fingerprints before utility degrades, yet scalability (many fingerprints) is needed to lower false discovery, resist leakage, and defend against collusion among model hosts.
Approach
Sample fingerprint keys at low temperature from the model so they are in-distribution, and choose each response uniformly from the k tokens just outside the top-t nucleus of the base model's next-token distribution, so responses are unlikely but sensible; regularize fine-tuning with weight averaging and data mixing. Also assigns fingerprints randomly to models for collusion-resistant identificatio
Key results
- Perinucleus fingerprinting adds 24,576 fingerprints to Llama-3.1-8B with minimal OpenLLM degradation, about two orders of magnitude more than ENGLISH-RANDOM which degrades after ~256 fingerprints.
- Relative OpenLLM performance stays above 95% at 8192 fingerprints across the 10 tested models, with relative drop under 5%.
- Persistence after 2 epochs of Alpaca SFT retains over 60% of an initial set of 8192 Perinucleus fingerprints.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Perinucleus responses instead of random unrelated responses with English keys | Fingerprinting Llama-3.1-8B · OpenLLM utility as fingerprint count increases | utility | Helped |
| Use Perinucleus responses instead of random unrelated responses with English keys | Fingerprinting Llama-3.1-8B · After SFT on Alpaca | persistence | Helped |
| Increase the number of Perinucleus fingerprints to 8192 | Llama-3.1-8B · OpenLLM | relative performance | Hurt |
| Increase Perinucleus threshold above 0.9 (ablation) | Perinucleus fingerprints · Llama-3.1-8B OpenLLM | utility | Hurt |
| Increase Perinucleus width beyond 10 (ablation) | Perinucleus fingerprints · Llama-3.1-8B OpenLLM | utility | Hurt |
| Add weight deviation penalty and data-mixing regularization (ablation) | Fingerprint training · OpenLLM harmlessness | utility | Helped |
| Use Perinucleus fingerprint design rather than baseline fingerprint designs (ablation) | Fingerprint training · After SFT | persistence | Helped |
| Fine-tune Perinucleus fingerprints on MathInstruct instead of Alpaca (ablation) | Llama-3.1-8B fingerprinted model · After SFT | persistence | Helped |
| Add one epoch of DPO on Orca pairs after Alpaca instruction tuning (ablation) | Llama-3.1-8B fingerprinted model · After post-training | persistence | No clear effect |
| Increase the number of SFT samples (ablation) | Perinucleus fingerprints · Llama-3.1-8B after SFT | persistence | Hurt |
| Increase SFT epochs (ablation) | Perinucleus fingerprints · Llama-3.1-8B after Alpaca SFT | persistence | Hurt |
| Use multiple fingerprint responses per key instead of one (ablation) | Perinucleus fingerprints under increased sampling temperature · Fingerprint detection | detection probability | Helped |
Surprising findings
- RANDOM keys are highly persistent after SFT because they are out-of-distribution, though they are easily detected and filtered by adversaries.
- SFT on MathInstruct causes less forgetting of fingerprints than Alpaca, and additional DPO adds little extra forgetting.
Limitations
- Persistence decreases as the number of fingerprints grows, with about 60% retained at 8192 fingerprints after SFT.
- Some fingerprints produce unusual completions when the response token had very low base-model probability.
- Collusion-resistance bound scales as 2^K K^(K+1), which is poor in coalition size K; adaptive attacks on sampling are left for future work.
Experimental setup
Datasets: Alpaca MathInstruct Orca pairs tinyBenchmarks
Benchmarks: OpenLLM MMLU TruthfulQA GSM8K Winogrande HellaSwag ARC-C
Metrics: average OpenLLM score persistence (fraction of fingerprints recalled after SFT) detection rate under collusion
Baselines: RANDOM ENGLISH-RANDOM
Builds on: nucleus sampling ENGLISH-RANDOM fingerprinting continual learning weight deviation penalty
Closest papers by meaning
- Queries, Representation & Detection: The Next 100 Model Fingerprinting Schemes (AAAI 2025 · similarity 0.89)
- HuRef: HUman-REadable Fingerprint for Large Language Models (NeurIPS 2024 · similarity 0.86)
- Fingerprinting Inference Systems of Large Language Models (NeurIPS 2026 · similarity 0.86)
- OmniMark: Efficient and Scalable Latent Diffusion Model Fingerprinting (AAAI 2025 · similarity 0.85)
- Intersecting-Boundary-Sensitive Fingerprinting for Tampering Detection of DNN Models (ICML 2024 · similarity 0.85)
- SIF: Semantically In-Distribution Fingerprints for Large Vision-Language Models (CVPR 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.