Research map / Architectures and efficiency / Pruning and efficient tuning
NUDGE: Lightweight Non-Parametric Fine-Tuning of Embeddings for Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NUDGE, non-parametric embedding fine-tuning that directly moves data embeddings toward training queries with closed-form, validation-tuned step sizes, outperforming fine-tuning and adaptor baselines on text and image retrieval.
Problem
Improving k-NN retrieval accuracy on a specific dataset and query workload by fine-tuning embeddings, without the cost and overfitting of fine-tuning the pre-trained model or training adaptors.
Approach
Treat the data embeddings themselves as parameters and move each data embedding toward the sum of training queries for which it is the ground-truth answer, with bounded (NUDGE-M) or unit-norm-constrained (NUDGE-N) changes whose magnitude is chosen in closed form by maximizing validation accuracy.
Key results
- NUDGE-N improves average text NDCG@10 from 48.7 to 61.1 with BGE-S (+12.4), versus +3.8 for PTFT and +2.9 for Adapter.
- On image retrieval, NUDGE-N raises CLIP-B NDCG@10 by +14.3 over no fine-tuning, while Adapter gives +3.0.
- NUDGE-M and NUDGE-N run in 1.14 and 2.18 minutes on GPU versus 447 minutes for PTFT and 7.99 minutes for Adapter.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply NUDGE-N instead of no fine-tuning | BGE-S · Text retrieval datasets (average) | NDCG@10: 48.7 -> 61.1 | Helped |
| Apply NUDGE-N instead of Adapter | BGE-S · Text retrieval datasets (average) | NDCG@10: 51.6 -> 61.1 | Helped |
| Apply NUDGE-N instead of no fine-tuning | GTE-L · Text retrieval datasets (average) | NDCG@10: 53.3 -> 62.7 | Helped |
| Apply NUDGE-N instead of no fine-tuning | TE3-L · Text retrieval datasets (average) | NDCG@10: 52.2 -> 63.9 | Helped |
| Apply NUDGE-N instead of no fine-tuning | CLIP-B · Image retrieval datasets (average) | NDCG@10: 26.9 -> 41.1 | Helped |
| Apply NUDGE-N instead of no fine-tuning | CLIP-L · Image retrieval datasets (average) | NDCG@10: 32.5 -> 43.3 | Helped |
| Add normalized-embedding constraint during optimization (ablation) | NUDGE-M on BGE-S · Text retrieval datasets (average) | NDCG@10: 57.0 -> 61.1 | Helped |
| Normalize embeddings after optimization rather than constrain normalization during optimization (ablation) | NUDGE-M on BGE-S · Text retrieval datasets (average) | NDCG@10: 61.1 -> 58.8 | Hurt |
| Remove the constraint on the magnitude of embedding changes (ablation) | NUDGE-N on BGE-S · Text retrieval datasets (average) | NDCG@10: 61.1 -> 56.6 | Hurt |
| Use unnormalized embeddings rather than normalized embeddings (ablation) | NUDGE-N on BGE-S · Out-of-distribution queries | NDCG@10: 51.2 -> 38.0 | Hurt |
Surprising findings
- Unbounded-magnitude variant (NUDGE-NU) performs worse than all other variants, showing the benefit of bounding changes.
- Unnormalized NUDGE-M degrades out-of-distribution NDCG@10 by 10 points below no fine-tuning, while NUDGE-N improves it.
Limitations
- NUDGE-M shows degraded out-of-distribution performance because unnormalized embeddings can distort answers for distant queries.
- Gains depend on the pre-trained model and dataset; ArguAna shows no gain since counterargument retrieval needs larger systematic changes.
- Unconstrained embedding fine-tuning is NP-hard, so the method relies on constrained variants and validation-set tuning of the step size.
Experimental setup
Datasets: NF-Corpus SciFact ArguAna Fever NQ TriviaQA HotpotQA COCO Flickr
Benchmarks: NF-Corpus SciFact ArguAna Fever NQ TriviaQA HotpotQA COCO Flickr
Metrics: NDCG@10 Recall@1 Recall@10
Baselines: No Fine-Tuning Adaptor PTFT PTFT-L Adaptor-L
Builds on: Adaptors (linear adaptors) Multiple Negative Ranking loss
Closest papers by meaning
- NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models (ICLR 2025 · similarity 0.90)
- MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interaction (ICLR 2026 · similarity 0.89)
- Your Embedding Model Is SMARTer Than You Think (NeurIPS 2026 · similarity 0.88)
- U-MARVEL: Unveiling Key Factors for Universal Multimodal Retrieval via Embedding Learning with MLLMs (ICLR 2026 · similarity 0.88)
- Contextual Document Embeddings (ICLR 2025 · similarity 0.88)
- Exploiting Distribution Constraints for Scalable and Efficient Image Retrieval (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.