Research map / Architectures and efficiency / Attention and transformers

In-context Learning in Presence of Spurious Correlations

NeurIPS 2026 · Architectures and efficiency › Attention and transformers · approach: Training dynamics of in-context learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Trains transformer in-context learners for spurious-feature classification by permuting embedding dimensions and using intermediate queries, showing single-task ICL learners match ERM/GroupDRO but generalize poorly, and that diverse synthetic iNaturalist tasks yield a more general learner.

Problem

Conventional in-context learner training on classification with spurious features leads to in-weights learning (ignoring context) and non-robustness to shifts in label-spurious correlation; it is unclear whether a robust classifier can be learned purely via meta-training.

Approach

Randomly permute input embedding dimensions per sequence to induce in-context learning from a single task, and form ICL sequences with interleaved intermediate queries drawn from a group-balanced distribution (with custom attention mask and position indices) so that context representations are agnostic to queries, simulating distribution shift in-context.

Key results

• On Waterbirds, Proposed+P reaches about 91.95% worst-group accuracy at context 512, versus ERM 84.23% and GroupDRO 92.43% (reported in text); 1-NN gets 90.03%.
• Single-task Naive ICL without permutation stays near 84-85% worst-group accuracy across context sizes, indicating in-weights learning; with permutation it rises to 92.60% at context 512 for Naive+P.
• Training on diverse synthetic iNaturalist tasks (G+P+I) is on par with or better than 1-NN, ERM and GroupDRO from context length 32 on OOD tasks, but fails on Waterbirds-severe.

Surprising findings

• Naive ICL trained on a single task ignores context: worst-group accuracy at 512 context equals that at 2 examples, i.e., in-weights learning.
• The single-task learned algorithm swapped-label and background-prediction tasks show it encodes the specific spurious feature, and it fails completely (0% minority accuracy) on Waterbirds-severe.

Limitations

• Training high-dimensional ICL transformers is computationally costly.
• Only one model size, width and depth were explored.
• Generalization fails under strong, dense additive spurious features (Waterbirds-severe); spurious annotations are required for training.

Experimental setup

Datasets: Waterbirds iNaturalist CelebA MultiNLI CUB-200

Benchmarks: Waterbirds Waterbirds-severe iNaturalist OOD test CelebA MultiNLI CUB-200

Metrics: worst-group accuracy majority-group accuracy minority-group accuracy overall accuracy

Baselines: 1-NN ERM GroupDRO Naive ICL approach

Builds on: in-context learning transformers (regression-style ICL training) GroupDRO induction heads

Closest papers by meaning

• In-Context Learning with Representations: Contextual Generalization of Trained Transformers (NeurIPS 2024 · similarity 0.91)
• Toward Understanding In-context vs. In-weight Learning (ICLR 2025 · similarity 0.89)
• Transformers are Minimax Optimal Nonparametric In-Context Learners (NeurIPS 2024 · similarity 0.89)
• The mechanistic basis of data dependence and abrupt learning in an in-context classification task (ICLR 2024 · similarity 0.88)
• Probing the Decision Boundaries of In-context Learning in Large Language Models (NeurIPS 2024 · similarity 0.88)
• In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitzness (NeurIPS 2024 · similarity 0.88)