Research map / Large language models / Instruction and fine-tuning
What's New in My Data? Novelty Exploration via Contrastive Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces novelty discovery through generation and Contrastive Generative Exploration (CGE), which contrasts a pre-trained model with its fine-tuned version (plus an iterative variant) to generate examples revealing novel data content such as non-English text, toxic text, and source code, without dataset access.
Problem
Fine-tuning datasets are massive, noisy, and confidential, so developers cannot inspect them directly, yet they need to know novel properties (languages, toxicity, code) the fine-tuning introduced.
Approach
Use contrastive decoding between the fine-tuned and pre-trained model log-probabilities to generate examples that highlight what fine-tuning added; an iterative variant updates the pre-trained model on previously generated examples so later generations cover different novelties.
Key results
- Contrastive score reaches AUROC 0.99 (FPR95 0.05) for detecting non-English text in OpenLLaMA fine-tuning data, versus 0.96 for the best baseline Prob_ft on that setting.
- CGE static achieves 0.99 detection rate for non-English text on OpenLLaMA vs 0.28 for sampling from the fine-tuned model; the iterative version reaches 0.82 coverage vs 0.65 for sampling.
- Under DP-Adam with LoRA, coverage stays above 60% for both models across noise multipliers up to 0.8.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CGE static instead of sampling from the fine-tuned model | OpenLLaMA-3B · Non-English text | Detection: 0.28±0.03 -> 0.99±0.01 | Helped |
| Use CGE static instead of sampling from the fine-tuned model | OpenLLaMA-3B · Toxic text | Detection: 0.33±0.05 -> 0.78±0.06 | Helped |
| Use CGE static instead of sampling from the fine-tuned model | Falcon-RW-1B · Non-English text | Detection: 0.01±0.00 -> 0.53±0.05 | Helped |
| Use CGE static instead of sampling from the fine-tuned model | Falcon-RW-1B · Source code | Detection: 0.06±0.03 -> 0.92±0.02 | Helped |
| Use the iterative version instead of static CGE (ablation) | OpenLLaMA-3B · Non-English text | Coverage: 0.55±0.05 -> 0.82±0.11 | Helped |
| Use the iterative version instead of static CGE (ablation) | OpenLLaMA-3B · Non-English text | Detection: 0.99±0.01 -> 0.18±0.03 | Hurt |
| Use the iterative version instead of static CGE (ablation) | OpenLLaMA-3B · Toxic text | Coverage: 0.75±0.05 -> 0.83±0.08 | Helped |
| Use the iterative version instead of static CGE (ablation) | Falcon-RW-1B · Source code | Coverage: 0.62±0.04 -> 0.90±0.07 | Helped |
| Use sampling instead of beam sampling (ablation) | Static CGE on OpenLLaMA-3B · Non-English text | Coverage: 0.55±0.05 -> 0.70±0.07 | Helped |
| Set alpha to 0.1 instead of 0.01 (ablation) | Static CGE on OpenLLaMA-3B · Non-English text | Coverage: 0.55±0.05 -> 0.25±0.05 | Hurt |
| Set alpha to 0.1 instead of 0.01 (ablation) | Iterative CGE on OpenLLaMA-3B · Non-English text | Coverage: 0.82±0.11 -> 0.62±0.04 | Hurt |
Surprising findings
- Negative log-probability under the pre-trained model fails to detect non-English text, since multi-byte character tokens can receive high probability.
- The iterative version trades detection rate for coverage: its detection rate drops to 0.18 on OpenLLaMA non-English text while coverage rises.
Limitations
- Trade-off between detection rate and coverage in the iterative version
- Experiments use synthetic fine-tuning mixtures with distinct novel domains rather than real-world confidential data
- Hyperparameters fixed from one setting; no validation set with ground-truth novelties
Experimental setup
Datasets: RedPajama ToxiGen Wikipedia (10 non-English languages) GitHub Code RefinedWeb
Benchmarks: ToxiGen GitHub Code Wikipedia non-English
Metrics: AUROC FPR95 detection rate coverage rate
Baselines: MSP Energy GradNorm NegativeProb (pt) Prob (ft) Sampling from fine-tuned models
Builds on: contrastive decoding adaptive plausibility constraint dataset distillation gradient norm OOD detection DP-Adam
Closest papers by meaning
- Towards a Theoretical Understanding of Synthetic Data in LLM Post-Training: A Reverse-Bottleneck Perspective (ICLR 2025 · similarity 0.84)
- Diversity as a Reward: Fine-Tuning LLMs on a Mixture of Domain-Undetermined Data (NeurIPS 2025 · similarity 0.84)
- DataGen: Unified Synthetic Dataset Generation via Large Language Models (ICLR 2025 · similarity 0.83)
- Get more for less: Principled Data Selection for Warming Up Fine-Tuning in LLMs (ICLR 2024 · similarity 0.83)
- Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferences (NeurIPS 2024 · similarity 0.83)
- From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Data (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.