Research map / Large language models / Instruction and fine-tuning
TART: A plug-and-play Transformer module for task-agnostic reasoning
Kush Bhatia, Avanika Narayan, Christopher De Sa, Christopher Re
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use TART reasoning module on top of base LLM embeddings | LLM in-context learning · 14 NLP tasks | accuracy: +18.4 points | Improved |
| TART vs adapter heads | TART · 14 NLP tasks | accuracy: +3.4 points | Improved |
| TART vs full fine-tuning | TART · 14 NLP tasks | accuracy: within 3.1 points | Hurt |
| vanilla to leave-one-out (LOO) embeddings (ablation) | TART · NLP tasks | accuracy | Improved |
| scale in-context examples from 18 to 64 | TART · 14 NLP tasks | accuracy: 4.8% vs 9.0% for full fine-tuning | Improved |
| scale base model from 100M to 6B | TART · GPT-Neo family | accuracy: +9.8% | Improved |
| scale base model from 100M to 6B | TART vs fine-tuning · GPT-Neo family | accuracy gap: 7.5% -> 2.2% | Improved |
| scale in-context examples from 18 to 256 | TART · long-text NLP tasks | accuracy: +6.8% | Improved |
| TART vs in-context learning beyond context limit | TART · long-text NLP tasks | accuracy: 19.1% | Improved |
| TART with GPT-Neo 125M vs Bloom 176B | TART · RAFT | accuracy: 0.595 -> 0.634 | Improved |
| TART with GPT-Neo 125M vs GPT-3 175B | TART · RAFT | accuracy: 0.673 -> 0.634 | Hurt |
| task-specific fine-tuning on AGNews | LLM · other tasks | accuracy: 25.77% | Hurt |
Closest papers by meaning
- Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes (ICLR 2026 · similarity 0.90)
- Training Language Models to Reason Efficiently (NeurIPS 2025 · similarity 0.89)
- General-Reasoner: Advancing LLM Reasoning Across All Domains (NeurIPS 2025 · similarity 0.89)
- Supervised Knowledge Makes Large Language Models Better In-context Learners (ICLR 2024 · similarity 0.89)
- Native Reasoning Models: Training Language Models to Reason on Unverifiable Data (ICLR 2026 · similarity 0.89)
- The Generalization and Robustness of Transformer-Based Language Models on Commonsense Reasoning (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
