Research map / Graph learning / GNN architectures
Noisy Node Classification by Bi-level Optimization Based Multi-Teacher Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BO-NNC, a multi-teacher distillation method for noisy node classification that learns a teacher weight matrix via bi-level optimization and adds a label improvement module.
Problem
Node classification on graphs with noisy labels, where single-model methods are fragile at high noise rates and teacher-weighting methods need many clean labels.
Approach
Builds diverse teachers from several self-supervised graph encoders (GCA, DGI, SUGRL) with MLP classifiers, fuses their predictions via a learned teacher weight matrix, and updates that matrix in an upper-level step using clean nodes selected by teacher consensus and embedding similarity, with student feedback via unrolled differentiation; a label improvement module filters noisy labels and adds p
Key results
- At 60% uniform noise, BO-NNC reaches 73.16% on Cora, 58.18% on Citeseer, 81.95% on DBLP, 87.01% on Photo, and 80.85% on Computers, the best among compared methods.
- Ablation: with all three components, average gain of 5.01% over single-component variants at the highest noise rates.
- Versus co-attention weighting, BO-NNC improves by 0.86% (uniform 60%) and 1.88% (pair 40%) on average.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add student integration, bi-level multi-teacher distillation, and label improvement together (ablation) | BO-NNC · Cora, uniform 60% noise | accuracy (%): 55.22 (1.0) -> 73.16 (1.0) | Helped |
| add label improvement to student integration and bi-level distillation (ablation) | BO-NNC · Cora, uniform 60% noise | accuracy (%): 58.56 (5.0) -> 73.16 (1.0) | Helped |
| add bi-level distillation to student integration and label improvement (ablation) | BO-NNC · Cora, pair 40% noise | accuracy (%): 73.36 (3.5) -> 77.50 (2.9) | Helped |
| replace mean teacher fusion with bi-level optimization (ablation) | BO-NNC · Cora, uniform 60% noise | accuracy (%): 71.26 (1.4) -> 73.16 (1.0) | Helped |
| replace co-attention teacher fusion with bi-level optimization (ablation) | BO-NNC · Cora, pair 40% noise | accuracy (%): 72.96 (0.4) -> 77.50 (2.9) | Helped |
| use three teacher models instead of one teacher model (ablation) | BO-NNC · uniform and pair noise, highest noise rates | accuracy (%): +2.68% uniform; +2.49% pair | Helped |
| increase r when r is less than 0.4 (ablation) | BO-NNC · 60% noise rate | accuracy (%) | Helped |
| increase r when r is larger than 0.6 (ablation) | BO-NNC · 60% noise rate | accuracy (%) | Hurt |
| vary rho below 70 (ablation) | BO-NNC · 60% noise rate | accuracy (%) | No clear effect |
| increase rho above 70 (ablation) | BO-NNC · 60% noise rate | accuracy (%) | Hurt |
| vary beta1, beta2, or alpha (ablation) | BO-NNC · 60% noise rate | accuracy (%) | No clear effect |
| use BO-NNC instead of DGI | node classification across datasets, uniform noise · uniform noise at 20%, 40%, and 60% | accuracy (%): 2.93%; 4.99%; 13.44% | Helped |
Surprising findings
- Mean fusion of teacher predictions is only slightly worse than co-attention, and the gain of bi-level optimization over co-attention is modest (under 2%).
Limitations
- Hyperparameter sensitivity to r and rho; performance degrades when too many clean labels are removed or too many incorrect pseudo-labels are selected.
- Bi-level optimization relies on enough clean nodes; the clean-node selection depends on teacher consensus and embedding similarity.
Experimental setup
Datasets: Cora Citeseer DBLP Photo Computers
Benchmarks: Cora Citeseer DBLP Photo Computers
Metrics: classification accuracy
Baselines: GCN GAT DGI GCA SUGRL GREET JoCoR NRGNN MTS-GNN
Builds on: MTS-GNN co-attention multi-teacher distillation DGI GCA SUGRL
Closest papers by meaning
- Training Graph Transformers via Curriculum-Enhanced Attention Distillation (ICLR 2024 · similarity 0.88)
- Robust Node Classification on Graph Data with Graph and Label Noise (AAAI 2024 · similarity 0.86)
- Label Attentive Distillation for GNN-Based Graph Classification (AAAI 2024 · similarity 0.86)
- NoisyGL: A Comprehensive Benchmark for Graph Neural Networks under Label Noise (NeurIPS 2024 · similarity 0.86)
- GC4NC: A Benchmark Framework for Graph Condensation on Node Classification with New Insights (NeurIPS 2025 · similarity 0.85)
- Joint Graph Rewiring and Feature Denoising via Spectral Resonance (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.