Research map / Large language models / Instruction and fine-tuning
Cappy: Outperforming and Boosting Large Multi-Task LMs with a Small Scorer
Bowen Tan, Yun Zhu, Lijuan Liu, Eric Xing, Zhiting Hu, Jindong Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace self-scoring with Cappy-large selection | frozen FLAN-T5-XXL with 17 samples · BIG-Bench | Rouge-L: 32.02 -> 36.93 | Improved |
| increase samples from 4 to 17 for self-scoring | self-scoring on FLAN-T5-XXL · BIG-Bench | Rouge-L: 31.15 -> 32.02 | No clear effect |
| increase samples from 4 to 17 for Cappy | Cappy-large on FLAN-T5-XXL · BIG-Bench | Rouge-L: 33.64 -> 36.93 | Improved |
| add Cappy-large to finetuned FLAN-T5-Small | Cappy-large · BIG-Bench | Rouge-L: 33.95 -> 39.74 | Improved |
| add Cappy-large to finetuned FLAN-T5-Large | Cappy-large · BIG-Bench | Rouge-L: 46.49 -> 48.98 | Improved |
| add Cappy-large on top of in-context learning | FLAN-T5-Large ICL · BIG-Bench | Rouge-L: 27.00 -> 31.84 | Improved |
| add Cappy-large on top of prompt tuning | FLAN-T5-Large prompt tuning · BIG-Bench | Rouge-L: 38.43 -> 42.71 | Improved |
| Cappy scoring versus FLAN-T5-XXL self-scoring on codenames, simple_arithmetic_json, sufficient_information | Cappy · BIG-Bench per-task | Rouge-L | Hurt |
Closest papers by meaning
- Benchmarking Optimizers for Large Language Model Pretraining (NeurIPS 2026 · similarity 0.88)
- Enhancing Large Language Model Performance with Gradient-Based Parameter Selection (AAAI 2025 · similarity 0.88)
- BenTo: Benchmark Reduction with In-Context Transferability (ICLR 2025 · similarity 0.88)
- LensLLM: Unveiling Fine-Tuning Dynamics for LLM Selection (ICML 2025 · similarity 0.88)
- Two-stage LLM Fine-tuning with Less Specialization and More Generalization (ICLR 2024 · similarity 0.88)
- LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
