Research map / Probabilistic and scientific modeling / Causal discovery
How Few-Shot Examples Add Up: A Causal Decomposition of Function Vectors in In-Context Learning
Entang Wang, Yiwei Wang, Aleksandra Bakalova, Michael Hahn
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| approximate n-shot FV as OLS linear combination of per-example sub-FVs | function vectors · 10 ICL tasks, gemma-2-2b | cosine similarity / R2: mean cosine ≥0.925, mean R2 ≥0.875 | Improved |
| inject OLS-reconstructed FV instead of full FV | function vectors · 0-shot injection | injection accuracy ratio | Improved |
| remove cross-example attention (uncontextualized) (ablation) | gemma-2-2b FV heads · Ambiguous datasets | attention entropy: entropy reduction averaging -0.08 to -0.15 with contextualization | Mixed |
| add contextualization on normal tasks (ablation) | FV head attention · 10-shot normal tasks | attention mass center shift: ΔC ≈ -0.4 to -0.6 | Improved |
| add contextualization (ablation) | FV head attention · Present-Past Ambiguous 10-shot | share of attention on unambiguous examples: 32% -> 61% | Improved |
| patch ambiguous or unambiguous example Keys (ablation) | FV heads · Present-Past Ambiguous 5-shot | attention allocation | Hurt |
| contextualize Query-Key pathway (Shapley) (ablation) | function vectors · ICL tasks, gemma and Llama | FV injection accuracy | Improved |
| contextualize Value pathway (Shapley) (ablation) | function vectors · ICL tasks, gemma and Llama | FV injection accuracy | Mixed |
| corrupt demonstrations rather than query (ablation) | function vectors · Present-Past Ambiguous 10-shot | FV injection accuracy: 0.52 -> 0.19 | Hurt |
| corrupt query rather than demonstrations (ablation) | function vectors · Capitalization Ambiguous 10-shot | FV injection accuracy: 0.93 -> 0.82 | Hurt |
Closest papers by meaning
- Do different prompting methods yield a common task representation in language models? (NeurIPS 2025 · similarity 0.89)
- Capturing In-Context Learning Dynamics with Task Operators (NeurIPS 2026 · similarity 0.89)
- Disentangling Latent Shifts of In-Context Learning with Weak Supervision (NeurIPS 2025 · similarity 0.87)
- In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering (ICML 2024 · similarity 0.87)
- Function Vectors in Large Language Models (ICLR 2024 · similarity 0.87)
- Many-Shot In-Context Learning (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
