Research map / Large language models / Language, safety and interpretability
MAPLE: Many-Shot Adaptive Pseudo-Labeling for In-Context Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MAPLE, an influence-based many-shot ICL framework that selects impactful unlabeled samples for LLM pseudo-labeling and adaptively picks labeled and pseudo-labeled demonstrations per test query using graph-based node influence.
Problem
Many-shot in-context learning requires large labeled demonstration sets, which are costly; naively using unlabeled samples as demonstrations gives marginal and unstable gains.
Approach
Build a labeled-unlabeled kNN graph, score unlabeled nodes by a shortest-path-based node influence lower bound (number of shortest paths and average distance) to the labeled set, pseudo-label the top-P, then per test query build a graph over labeled and pseudo-labeled demonstrations and select the most influential ones as the many-shot prompt.
Key results
- On Banking77 with Gemini 1.5 Flash, MAPLE averages 78.5% versus 77.0% for RAG across pseudo-label budgets of 20 to 100.
- On Date with Gemini 1.5 Flash, MAPLE averages 55.8% versus 53.8% for Random and 53.6% for RAG.
- On GPQA with Gemini 1.5 Pro, MAPLE averages 44.3% versus 42.9% for RAG.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MAPLE instead of the strongest listed baseline | Gemini 1.5 Flash · Banking77 | accuracy (average across pseudo-label budgets): 77.0 -> 78.5 | Helped |
| Use MAPLE instead of the strongest listed baseline | Gemini 1.5 Pro · Banking77 | accuracy (average across pseudo-label budgets): 78.3 -> 80.3 | Helped |
| Use MAPLE instead of the strongest listed baseline | Gemini 1.5 Flash · Date | accuracy (average across pseudo-label budgets): 53.8 -> 55.8 | Helped |
| Use MAPLE instead of the strongest listed baseline | Gemini 1.5 Pro · Date | accuracy (average across pseudo-label budgets): 67.0 -> 68.0 | Helped |
| Use MAPLE instead of the strongest listed baseline | Gemini 1.5 Flash · GPQA | accuracy (average across pseudo-label budgets): 34.7 -> 37.1 | Helped |
| Use MAPLE instead of the strongest listed baseline | Gemini 1.5 Pro · GPQA | accuracy (average across pseudo-label budgets): 42.9 -> 44.3 | Helped |
| Remove adaptive demonstration selection and use fixed demonstrations (ablation) | MAPLE · XSum | ROUGE-L | Mixed |
| Use a KV cache with fixed demonstrations (ablation) | MAPLE without adaptive selection · XSum | inference time: 1.93 -> 1.89; 3.74 -> 3.57; 4.50 -> 4.08 | Helped |
| Use a KV cache with fixed demonstrations (ablation) | MAPLE without adaptive selection · Date | inference time: 0.91 -> 0.94; 1.99 -> 1.92; 2.04 -> 1.96 | Mixed |
| Replace Contriever with SBert (ablation) | MAPLE · Date | accuracy: 51.4 -> 52.7; 52.4 -> 54.0; 54.4 -> 55.2 | Helped |
| Replace Contriever with DeBERTa (ablation) | MAPLE · Date | accuracy: 52.0 -> 54.4; 53.6 -> 55.2; 55.2 -> 57.6 | Helped |
| Replace Contriever with SBert (ablation) | MAPLE · GoEmotion | accuracy: 31.3 -> 34.7; 32.7 -> 36.7; 33.3 -> 37.3 | Helped |
Surprising findings
- Performance on Tracking7 and Salient drops as more pseudo-labeled demonstrations are added, attributed to noisy pseudo-labels and irrelevant examples.
- KV cache gives only marginal inference-time savings for API-based models such as Gemini 1.5 Flash.
Limitations
- Gains are limited on some datasets (Tracking7, XSum) where pseudo-label quality is low.
- Relies on API-based Gemini models; the adaptive selection makes per-query prompts that cannot use KV caching.
- Influence score theory relies on simplifying assumptions (linear GCN, identity weights, uniform degree expectation).
Experimental setup
Datasets: XSum Date Salient Tracking7 Financial PhraseBank Banking77 GoEmotion GPQA
Baselines: Zero-shot Few-shot Random pseudo-labeling RAG RAG-Adapt
Builds on: Contriever GCN EPR CEIL IDS
Closest papers by meaning
- Many-Shot In-Context Learning (NeurIPS 2024 · similarity 0.92)
- From Few to Many: Self-Improving Many-Shot Reasoners Through Iterative Optimization and Generation (ICLR 2025 · similarity 0.90)
- Revisiting In-context Learning Inference Circuit in Large Language Models (ICLR 2025 · similarity 0.90)
- In-Context Learning Learns Label Relationships but Is Not Conventional Learning (ICLR 2024 · similarity 0.89)
- MIR-Bench: Can Your LLM Recognize Complicated Patterns via Many-Shot In-Context Reasoning? (NeurIPS 2025 · similarity 0.89)
- Towards Auto-Regressive Next-Token Prediction: In-context Learning Emerges from Generalization (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.