Research map / Learning theory and algorithms / Kernels and regression theory
Efficient Estimation of Kernel Surrogate Models for Task Attribution
Zhenshuo Zhang, Minxuan Duan, Hongyang R. Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use kernel surrogate instead of linear surrogate | surrogate model · Modular arithmetic quadratic | LDS: 0.44 -> 0.52 | Improved |
| use kernel surrogate instead of linear surrogate | surrogate model · Modular arithmetic addition | LDS: 0.18 -> 0.30 | Improved |
| use kernel surrogate instead of linear surrogate | surrogate model · In-context learning Coin flip | LDS: 0.43 -> 0.54 | Improved |
| use kernel surrogate instead of linear surrogate | surrogate model · In-context learning SST-2 | LDS: 0.33 -> 0.37 | Improved |
| use kernel surrogate instead of linear surrogate | surrogate model · Meta-World MT10 | LDS: 0.76 -> 0.80 | Improved |
| select ICL prompts with kernel surrogate attribution instead of linear surrogate | task selection · ICL Coin flip | loss: 0.05 -> 0.02 | Improved |
| select ICL prompts with kernel surrogate attribution instead of linear surrogate | task selection · ICL SST-2 | loss: 0.20 -> 0.16 | Improved |
| co-train with top-3 tasks chosen by kernel surrogate instead of linear surrogate | task selection · Meta-World MT10 | target rewards: 13.1 -> 18.8 | Improved |
| increase number of prompt samples from 50 to 500 | kernel surrogate models · In-context learning | LDS: 0.31 on SST-2 and 0.53 on Coin-Flip | No clear effect |
| use SAM or NSO Hessian regularization instead of plain SGD (ablation) | linear surrogate model · Modular quadratic task | LDS: LDS dropped to 0.01 with SGD | Improved |
Closest papers by meaning
- BOTS: A Unified Framework for Bayesian Online Task Selection in LLM Reinforcement Finetuning (ICLR 2026 · similarity 0.88)
- Identifying Selections for Unsupervised Subtask Discovery (NeurIPS 2024 · similarity 0.87)
- Prediction-Powered Inference Across Many Tasks for AI Evaluations and Social Science Research (NeurIPS 2026 · similarity 0.87)
- AdaMerging: Adaptive Model Merging for Multi-Task Learning (ICLR 2024 · similarity 0.86)
- Towards Few-Shot Adaptation of Foundation Models via Multitask Finetuning (ICLR 2024 · similarity 0.86)
- Task Vector Geometry Underlies Dual Modes of Task Inference in Transformers (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
