Research map / Large language models / Instruction and fine-tuning
Model Provenance Testing for Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a black-box model provenance tester that compares next-token agreement rates between a suspect model and candidate parents versus control models, using z-tests with Holm-Bonferroni correction.
Problem
Determining whether a deployed LLM was derived (e.g., via fine-tuning) from a given foundation model, using only black-box query access and no knowledge of training data or procedures.
Approach
Derived models keep output-distribution similarity to their parent; the tester measures the fraction of prompts on which the first output token matches, compares it to control models with z-tests, and controls family-wise error via Holm-Bonferroni. Adds rejection sampling of prompts and best-arm identification to reduce queries.
Key results
- With 3,000 prompts per model, the tester achieves 90-95% precision and 80-90% recall across Bench-A and Bench-B.
- Rejection sampling with k=64 reduces required prompts from 3,000 to about 500 while keeping similar precision and recall.
- Best-arm identification reduces offline queries by 10-30% but cuts recall substantially (e.g., 0.86 to 0.63 on Bench-A at T=1000).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase the number of prompts | Provenance tester · Bench-A and Bench-B | recall: 80%-90% | Helped |
| Increase the number of prompts | Provenance tester · Bench-A and Bench-B | precision: approximately $0.95 | Hurt |
| Use entropy-based rejection prompt sampling instead of uniform sampling (ablation) | Provenance tester · Bench-A and Bench-B | online query complexity: from $3,000 to just $500 per model | Helped |
| Use BAI to reduce offline queries (ablation) | Provenance tester · Bench-A, 1,000-query budget | recall: 0.86 to 0.63 | Hurt |
| Use BAI to reduce offline queries (ablation) | Provenance tester · Bench-B, 1,000-query budget | recall: 0.68 to 0.42 | Hurt |
| Use a smaller randomly sampled control set (ablation) | Provenance tester · Bench-A, 4 control models | precision: 55% | Hurt |
| Use a smaller randomly sampled control set (ablation) | Provenance tester · Bench-B, 4 control models | precision: less than 10% | Hurt |
| Test with a known suspected parent rather than unspecified parents (ablation) | Provenance tester · Bench-A and Bench-B | precision: 100% | Helped |
| Vary the significance level (ablation) | Provenance tester · Bench-A | efficiency | No clear effect |
| Increase prompts from 1,000 to 3,000 | Provenance tester · Bench-A | true parent ranked highest by similarity: 93% | Helped |
| Increase prompts from 1,000 to 3,000 | Provenance tester · Bench-B | true parent ranked highest by similarity: 89% | Helped |
Surprising findings
- Precision decreases as the number of prompts increases, contrary to usual hypothesis-testing intuition, because more prompts detect non-provenance similarities.
- With a small random control set (4 models), precision on Bench-B falls below 10%, while Bench-A remains at 55%.
Limitations
- Ground truth for Bench-B relies partly on heuristic methods (naming, description, weight similarity), which are less reliable.
- Precision and recall depend heavily on control set size and quality, especially for diverse benchmarks.
- Only non-adaptive adversaries are evaluated; adaptive evasion is not studied.
Experimental setup
Datasets: Hugging Face models (Bench-A, Bench-B) LLM-generated prompts
Benchmarks: Bench-A Bench-B
Metrics: precision recall
Baselines: adversarial fingerprinting LLMmap
Builds on: Holm-Bonferroni method best arm identification
Closest papers by meaning
- Model Equality Testing: Which Model is this API Serving? (ICLR 2025 · similarity 0.86)
- Calibrating Large Language Models with Sample Consistency (AAAI 2025 · similarity 0.85)
- Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification (ICLR 2025 · similarity 0.85)
- TRACER: Verifiable Generative Provenance for Multimodal Tool-Using Agents (NeurIPS 2026 · similarity 0.85)
- Raising the Bar: Investigating the Values of Large Language Models via Generative Evolving Testing (ICML 2025 · similarity 0.85)
- Gorilla: Large Language Model Connected with Massive APIs (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.