Research map / Label-efficient and robust learning / Recognition and long-tail
TarGATE: Target-Aware Data Selection via Token-Attenuation Gates
Xiandi Luo, Shiwei Li, Haozhao Wang, Yihao Ouyang, Zhuoqi Hu, Yichen Li, Xiao Yang, Huning Liu, Ruixuan Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use joint optimization of reference loss and warmup loss (beta between 0 and 2) instead of only one objective (ablation) | TarGATE · CommonsenseQA, GSM8K, MBPP (Qwen3-0.6B selector, Qwen3-1.7B target) | accuracy | Improved |
| Use smaller beta for reasoning and coding tasks, larger beta for CommonsenseQA (ablation) | TarGATE · GSM8K, MBPP vs CommonsenseQA | accuracy | Mixed |
| Shrink reference set size (100% to 50% to 25%) and lower beta (ablation) | TarGATE · CQA, GSM8K, MBPP | accuracy | Mixed |
| Vary regularization weight lambda away from about 1 (ablation) | TarGATE · CQA, GSM8K, MBPP | accuracy | Hurt |
| Select data with TarGATE instead of training on the full 200k pool | Tulu-3 fine-tuning · Tulu-3 average (3 targets x 3 tasks) | average accuracy: 64.9 -> 68.2 | Improved |
| Use TarGATE instead of random selection in noisy pool | Synthetic noisy scenario · CQA/GSM8K average | average accuracy: 60.6 -> 66.5 | Improved |
| Use larger 1.7B selector instead of 0.6B selector (ablation) | TarGATE · Tulu-3 average | average accuracy: 68.2 -> 67.6 | Hurt |
| Use 1.7B selector instead of 0.6B for gradient baseline GradSim (ablation) | GradSim selecting for Llama3-8B · Tulu-3 GSM8K | accuracy: 56.3 -> 45.7 | Hurt |
| Replace gradient-based scoring with single-forward-pass IRR scoring | Data selection cost · 200k samples, Qwen3-1.7B | PetaFLOPs: 776.0 -> 424.9 | Improved |
Closest papers by meaning
- TAROT: Targeted Data Selection via Optimal Transport (ICML 2025 · similarity 0.85)
- Compute-Constrained Data Selection (ICLR 2025 · similarity 0.84)
- Computational Budget Should Be Considered in Data Selection (NeurIPS 2025 · similarity 0.84)
- STAFF: Speculative Coreset Selection for Task-Specific Fine-tuning (ICLR 2025 · similarity 0.84)
- TSDS: Data Selection for Task-Specific Model Finetuning (NeurIPS 2024 · similarity 0.83)
- LESS: Selecting Influential Data for Targeted Instruction Tuning (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
