Research map / Large language models / Reasoning and chain of thought

Many-Shot CoT-ICL: Making In-Context Learning Truly Learn

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Chain-of-thought training and analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Studies many-shot chain-of-thought in-context learning (CoT-ICL) on reasoning tasks, finds scaling is unstable for non-reasoning LLMs and order-sensitive, and proposes Curvilinear Demonstration Selection (CDS), which orders demonstrations to minimize embedding-space curvature.

Problem

Prior many-shot ICL scaling findings come from non-reasoning tasks; it is unclear whether many-shot chain-of-thought demonstrations reliably improve reasoning performance and how demonstration selection and ordering should work for them.

Approach

Frames many-shot CoT-ICL as in-context test-time learning with two principles (demonstrations should be understandable to the model; ordering should give smooth conceptual progression), and orders CoT demonstrations via a TSP heuristic minimizing embedding-trajectory turning angles (curvature).

Key results

• CDS yields up to a 5.42 percentage-point gain on a math task with 64 demonstrations.
• Ordering curvature correlates negatively with accuracy overall (r = -0.547) across three math tasks.
• Non-reasoning Llama 3.3 70B can show negative gains from adding CoT demonstrations, while QwQ-32B and DeepSeek-R1 improve with more demonstrations.

Surprising findings

• Similarity-based retrieval helps on BANKING77 but consistently hurts on reasoning tasks, with most-similar sets worse than dissimilar or original sets.
• Order variance across random permutations grows with the number of CoT demonstrations, unlike non-reasoning ICL where it shrinks.

Limitations

• Curvature-based ordering gains are smaller on number_theory where baselines are already high.
• Correlation analysis is observational; causality is supported only by a high-curvature ablation.
• Evaluation focuses on Qwen3 models for CDS; closed-source robustness is tested on a single model (gpt-5.2).

Experimental setup

Datasets: SuperGLUE NLU TREC BANKING77 GSM8K MATH DetectiveQA

Benchmarks: GSM8K MATH (geometry, number_theory, counting_and_probability) DetectiveQA BANKING77 TREC SuperGLUE NLU

Metrics: exact match accuracy standard deviation across orderings

Baselines: origin (dataset CoT ordering) most-dissimilar retrieval high-curvature ordering bge-m3 CDS variant

Builds on: chain-of-thought prompting retrieval-based demonstration selection Auto-CoT 2-opt local search

Closest papers by meaning

• Many-Shot In-Context Learning (NeurIPS 2024 · similarity 0.90)
• From Few to Many: Self-Improving Many-Shot Reasoners Through Iterative Optimization and Generation (ICLR 2025 · similarity 0.90)
• What Makes In-context Learning Effective for Mathematical Reasoning (ICML 2025 · similarity 0.90)
• Are Human-generated Demonstrations Necessary for In-context Learning? (ICLR 2024 · similarity 0.88)
• Don't Take Things Out of Context: Attention Intervention for Enhancing Chain-of-Thought Reasoning in Large Language Models (ICLR 2025 · similarity 0.87)
• Investigating the Pre-Training Dynamics of In-Context Learning: Task Recognition vs. Task Learning (ICLR 2025 · similarity 0.87)