Research map / Large language models / Instruction and fine-tuning
HuRef: HUman-REadable Fingerprint for Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HuRef, a black-box LLM fingerprinting method that computes invariant terms from the last layers of weights (robust to permutation/rotation attacks), encodes them into a Gaussian vector and then a StyleGAN2 dog image, with zero-knowledge proofs ensuring honest generation.
Problem
Identifying the base model of an LLM for copyright protection when parameters are altered by fine-tuning, continued pretraining, or RLHF, and parameters are not publicly accessible (black-box setting), while being robust to weight rearrangement attacks.
Approach
Observes that the direction of LLM parameter vectors stabilizes after pretraining and is preserved by subsequent training, then derives three Transformer-structure-invariant terms (M_a, M_b, M_f) robust to weight permutation and invertible-matrix rotation attacks; these are encoded by a contrastively trained CNN encoder with GAN-style Gaussianization into a StyleGAN2 image fingerprint, and ZKP pro
Key results
- Offspring models of 7 base LLMs show average ICS of 99.56, and LLaMA-7B offspring show average ICS 94.14 across RLHF, SFT, and continued pretraining.
- Identified the correct base model for all 51 offspring models (100% accuracy) among 18 base models, and 28 independently trained LLMs showed average ICS of 0.38.
- Fingerprint success rate of 100% across 13 LLaMA offspring models versus 11.94% average for Trap and 84.62% for IF (adapter, embedding-only); human subject study accuracy 94.74%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use invariant-term cosine similarity to identify offspring base models | HuRef · 51 offspring models from 18 base models | base-model identification accuracy: 100% | Helped |
| use HuRef instead of IF_adapter^2 | LLaMA-7B offspring · LLaMA-7B offspring | FSR: 84.62 -> 100.00 | Helped |
| use HuRef instead of Trap | LLaMA-7B offspring · LLaMA-7B offspring | FSR: 11.94 -> 100.00 | Helped |
| generate human-readable fingerprints from invariant terms | HuRef · Human subject study | accuracy: 94.74% | Helped |
| use invariant terms to compare independently trained models | HuRef · 28 independently trained LLMs | average ICS: 0.38 | Helped |
| add cosine-similarity suppression term L_A during Alpaca SFT (ablation) | Alpaca · Average across eight zero-shot benchmarks | accuracy: 62.54 -> 40.25; 33.38; 33.24; 33.52 | Hurt |
| add cosine-similarity suppression term L_A (PCS 97.13) (ablation) | Alpaca · BoolQ | accuracy: 77.49 -> 45.44 | Hurt |
| add cosine-similarity suppression term L_A (PCS 87.21) (ablation) | Alpaca · BoolQ | accuracy: 77.49 -> 42.23 | Hurt |
| add cosine-similarity suppression term L_A (PCS 80.23) (ablation) | Alpaca · BoolQ | accuracy: 77.49 -> 39.05 | Hurt |
| add cosine-similarity suppression term L_A (PCS 77.56) (ablation) | Alpaca · BoolQ | accuracy: 77.49 -> 41.62 | Hurt |
Surprising findings
- Fine-tuning Alpaca with an extra cosine-suppression loss drove the model to near-random zero-shot performance (average 33-40 vs 62.54), supporting the necessity of parameter direction.
- Trap and IF baselines achieved low success on many LLaMA offspring, showing existing LLM fingerprints are fragile.
Limitations
- Fingerprint images can cause minor misjudgments in human identification (94.74% accuracy), requiring ZKP-backed one-to-one comparison to compensate.
- One-to-one quantitative comparison is limited to open-source models, and the ZKP scheme is described at a high level without implementation.
- Evaluation covers a limited set of 51 offspring and 18 base models; invariance under more extreme post-training is not fully established.
Experimental setup
Datasets: Synthetic random Gaussian matrices (encoder training) The Pile (subset, for GPT-NeoX-350M) AFHQ dog dataset (StyleGAN2 pretraining)
Benchmarks: BoolQ PIQA HellaSwag WinoGrande ARC-e ARC-c RACE MMLU
Metrics: ICS (invariant-term cosine similarity) PCS zero-shot accuracy accuracy of base model identification FSR (fingerprint success rate) human subject accuracy
Baselines: Trap IF (adapter) IF (full parameters)
Builds on: StyleGAN2 Trap IF (instruction fingerprinting) GAN
Closest papers by meaning
- Scalable Fingerprinting of Large Language Models (NeurIPS 2025 · similarity 0.86)
- AttnDiff: Attention-based Differential Fingerprinting for Large Language Models (NeurIPS 2026 · similarity 0.86)
- MIA-Tuner: Adapting Large Language Models as Pre-training Text Detector (AAAI 2025 · similarity 0.86)
- Fingerprinting Inference Systems of Large Language Models (NeurIPS 2026 · similarity 0.86)
- WOUAF: Weight Modulation for User Attribution and Fingerprinting in Text-to-Image Diffusion Models (CVPR 2024 · similarity 0.85)
- Humanizing the Machine: Proxy Attacks to Mislead LLM Detectors (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.