Research map / Label-efficient and robust learning / Knowledge distillation
TrustAL: Trustworthy Active Learning Using Knowledge Distillation
Beong-woo Kwak, Youngwook Kim, Yu Jin Kim, Seung-won Hwang, Jinyoung Yeo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use consistency-aware predecessor selection instead of the standalone acquisition model | TrustAL-NC with CONF · TREC | correct consistency: 0.726 -> 0.774 | Improved |
| Use consistency-aware predecessor selection instead of the standalone acquisition model | TrustAL-NC with CONF · Movie review | correct consistency: 0.637 -> 0.676 | Improved |
| Use consistency-aware predecessor selection instead of the standalone acquisition model | TrustAL-NC with CONF · SST-2 | correct consistency: 0.686 -> 0.730 | Improved |
| Choose the consistency-aware predecessor rather than the most recent predecessor as teacher | TrustAL · TREC, Movie review, SST-2 | correct consistency: TREC: 0.743 -> 0.774; Movie review: 0.654 -> 0.676; SST-2: 0.705 -> 0.730 | Improved |
| Add predecessor-model distillation and consistency regularization | CONF, CORESET, and BADGE · TREC, Movie review, SST-2 | accuracy | Improved |
| Use TrustAL-NC instead of a standalone acquisition baseline | TrustAL with CONF, CORESET, or BADGE · TREC | labeled data needed to reach baseline convergence accuracy: 40%; 30%; 50% | Improved |
| Increase the distillation weight α from 0.75 to 10 (ablation) | TrustAL-NC with CONF · TREC stable phase | accuracy: 0.788({+0.011}) -> 0.810({+0.033}) | Improved |
| Increase the distillation weight α from 0.75 to 10 (ablation) | TrustAL-NC with CONF · TREC stable phase | MCI: 17.94({-5.83}) -> 10.62({-13.15}) | Improved |
| Set distillation weight α to 20 rather than 10 (ablation) | TrustAL-NC with CONF · TREC saturated phase | accuracy: 0.892({+0.059}) -> 0.874({+0.041}) | Hurt |
| Increase the distillation weight α (ablation) | TrustAL-NC · TREC | accuracy and MCI | Mixed |
| Use half-sized rather than the original development set (ablation) | TrustAL-NC · TREC | accuracy: 0.008 | Hurt |
| Add consistency distillation when acquired labels are corrupted | BADGE · TREC, after stable phase with 7% label noise | accuracy | Improved |
Closest papers by meaning
- Robust Active Distillation (ICLR 2023 · similarity 0.90)
- Continual Distillation of Teachers from Different Domains (CVPR 2026 · similarity 0.86)
- Distilling What Matters: Confidence-Aware Selective Distillation for Large Language Models (NeurIPS 2026 · similarity 0.85)
- Cooperative Knowledge Distillation: A Learner Agnostic Approach (AAAI 2024 · similarity 0.85)
- A Label is Worth A Thousand Images in Dataset Distillation (NeurIPS 2024 · similarity 0.85)
- Efficient Verified Unlearning For Distillation (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
