Research map / Label-efficient and robust learning / Recognition and long-tail
AIM: Manifold-based Data Filtering for Representation Finetuning
Qing Li, Qibin Zheng, Yi Liu, Xingchun Diao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use AIM manifold-based filtering instead of LESS | ReFT · Commonsense avg (LLaMA-3.2-3B) | average accuracy: 51.21 -> 54.62 | Improved |
| use AIM instead of LossSel | ReFT · Commonsense avg (LLaMA-2-7B) | average accuracy: 45.00 -> 47.42 | Improved |
| use AIM instead of Random on LLaMA-3-8B | ReFT · Commonsense avg (LLaMA-3-8B) | average accuracy: 64.12 -> 67.33 | Improved |
| use LossSel or LESS instead of random selection on LLaMA-3-8B commonsense | ReFT · Commonsense avg (LLaMA-3-8B) | average accuracy: 64.12 -> 61.71 (LossSel); 61.46 (LESS) | Hurt |
| use AIM instead of LossSel | ReFT · Arithmetic avg (LLaMA-3-8B) | average accuracy: 74.87 -> 76.67 | Improved |
| use AIM instead of Random | ReFT · Arithmetic avg (LLaMA-3.2-3B) | average accuracy: 60.10 -> 62.00 | No clear effect |
| use AIM-C classifier approximation instead of full AIM (ablation) | AIM · Arithmetic avg (LLaMA-2-7B) | average accuracy: 57.21 -> 57.62 | No clear effect |
| use AIM-C instead of full AIM (ablation) | AIM · Commonsense avg (LLaMA-3-8B) | average accuracy: 67.33 -> 62.08 | Hurt |
| weighted vs uniform segment aggregation, single segments (ablation) | AIM score · Commonsense selection precision | precision | Mixed |
| set explained variance threshold rho to 0.90 (ablation) | AIM · Commonsense selection precision | precision: peak at rho = 0.90 | Improved |
| ReFT vs LoRA under label noise | ReFT · Commonsense reasoning, LLaMA-3 8B | average accuracy under noise | Hurt |
Closest papers by meaning
- ReFT: Representation Finetuning for Language Models (NeurIPS 2024 · similarity 0.90)
- Tracing the Representation Geometry of Language Models from Pretraining to Post-training (NeurIPS 2025 · similarity 0.86)
- Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minima (NeurIPS 2026 · similarity 0.86)
- Bias-Restrained Prefix Representation Finetuning for Mathematical Reasoning (AAAI 2026 · similarity 0.85)
- Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models (ICML 2025 · similarity 0.85)
- The Geometry of Alignment Collapse: When Fine-Tuning Breaks Safety (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
