Research map / Label-efficient and robust learning / Test-time adaptation
Learning to Extrapolate to New Tasks: A Relational Approach to Task Extrapolation
Adam Ousherovitch, Yixin Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| decompose target into anchor plus transformation (RTE) instead of inductive prediction | MLP · Parameter extrapolation, Cubic | MSE: 2.86 -> 1.53 | Improved |
| decompose target into anchor plus transformation (RTE) | MLP · Parameter extrapolation, Sin Trend | MSE: 0.28 -> 0.051 | Improved |
| decompose target into anchor plus transformation (RTE) | MLP · Parameter extrapolation, Tri Trend | MSE: 0.46 -> 0.048 | Improved |
| decompose target into anchor plus transformation (RTE) | MLP · Parameter extrapolation, Exp | MSE: 1.25 -> 0.80 | Improved |
| use inferred anchor and transformation instead of ground-truth ones (ablation) | RTE · Parameter extrapolation, Tri Trend | MSE: 0.048 vs oracle 0.094 | No clear effect |
| add RTE to length extrapolation | MLP · Degree 9 polynomial | MSE: 0.5752 -> 0.3712 | Improved |
| use ground-truth anchor instead of inferred one (ablation) | RTE · Degree 9 polynomial | MSE: 0.3712 -> 0.0497 | Improved |
| add RTE to compositional extrapolation | MLP · Unseen compositions | aggregate MSE: 0.3891 -> 0.2867 | Improved |
| add relational fine-tuning with anchor and shift prompts | LoRA-tuned Qwen/Mistral · Sparse Parity |S|=6 | accuracy: 52.86% -> 66.07% | Improved |
| add RTE relational prompting | LoRA-tuned Qwen/Mistral · CodeIO | exact match accuracy: 19.8% -> 45.3% | Improved |
| compare RTE to CoT with majority vote (N=16) (ablation) | LLM · CodeIO | exact match accuracy: 30.2% -> 45.3% | Improved |
| provide ground-truth decomposition instead of inferred one (ablation) | RTE · CodeIO | exact match accuracy: 45.3% -> 76.0% | Improved |
Closest papers by meaning
- Towards Understanding Extrapolation: a Causal Lens (NeurIPS 2024 · similarity 0.86)
- Extrapolation by Association: Length Generalization Transfer In Transformers (NeurIPS 2025 · similarity 0.86)
- Task Vector Geometry Underlies Dual Modes of Task Inference in Transformers (NeurIPS 2026 · similarity 0.86)
- Does learning the right latent variables necessarily improve in-context learning? (ICML 2025 · similarity 0.85)
- Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder Decoder Perspective (ICML 2025 · similarity 0.85)
- Task Generalization with Autoregressive Compositional Structure: Can Learning from $D$ Tasks Generalize to $D^T$ Tasks? (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
