AI Paper Atlas

Research map / Label-efficient and robust learning / Knowledge distillation

BIRD: Behavior Induction via Representation-structure Distillation

Galen Pogoncheff, Michael Beyeler

ICLR 2026 · Label-efficient and robust learning › Knowledge distillation · approach: Few-shot and metric distillation · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
match teacher representation structure (CKA) instead of activation values (Hints) (ablation)BIRD · CIFAR-10->CIFAR-100, TinyImageNetaccuracy (%): e.g. MobileNetV2 C10->C100 51.53 -> 54.77Improved
use BIRD vs best baseline LwFrobustness transfer · C10->C100PGR (%): 16.2 -> 32.6Improved
linear probing or fine-tuning alonestudent models · robustness transferaccuracy (%): LP 10.95 vs none 51.31 (MobileNetV2 C10->C100)No clear effect
scale student up to ResNet-152 (25x teacher) (ablation)BIRD · weak-to-strong visionPGR (%): 22.4% PGR for ResNet-152Improved
add BIRD loss to soft-label distillation (ablation)GPT2-Large student · SciQPGR (%): 17.70 -> 24.19Improved
add BIRD loss to soft-label distillation (ablation)GPT2-Medium student · Cosmos QAPGR (%): 47.00 -> 42.76Hurt
add BIRD loss to soft-label distillation (ablation)GPT2 students · BoolQPGR (%): 0% PGR for bothNo clear effect
choose teacher by behavioral relevance, task relevance, complementary knowledgeteacher selection · 432 teacher-student pairs (ResNet50 student)variance explained in PGR: R2 81.8% TinyImageNet, 85.5% CIFAR-100, 73.6% CIFAR-10Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy