Research map / Graph learning / Graph self-supervised learning
Graph Your Own Prompt
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Graph Consistency Regularization (GCR), which inserts parameter-free Graph Consistency Layers that align batch-level feature similarity graphs with class-masked softmax prediction graphs across multiple depths, with adaptive layer weighting.
Problem
Intermediate feature representations of deep networks often carry noisy inter-class similarities that contradict the model's predicted semantics, hurting class separation and generalization.
Approach
Build a batch-level cosine feature similarity graph at several layers and align it (squared Frobenius norm on strictly upper-triangular parts) with a masked prediction graph that keeps only same-label pairs weighted by softmax-prediction similarity; layer weights are set adaptively from alignment discrepancy.
Key results
- On CIFAR-100, Late GCL raises mean accuracy across CNNs from 72.95% to 74.74% (+1.79 points).
- On ImageNet-1K, GCR lifts iFormer-S from 83.4% to 84.8% and iFormer-B from 84.6% to 86.1%.
- On Tiny ImageNet, best GCL configurations reach 54.32% vs 53.64% baseline mean accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add GCL at late stage | GCR · CIFAR-10 | accuracy (%): 93.32±2.26 -> 94.07±2.14 | Helped |
| Add GCL at late stage | GCR · CIFAR-100 | accuracy (%): 72.95±5.50 -> 74.74±5.73 | Helped |
| Add GCL at early stage | GCR · Tiny ImageNet | accuracy (%): 53.64±9.62 -> 54.32±9.39 | Helped |
| Add GCLs after six network layers | Kaggle cats vs. dogs model · Kaggle cats vs. dogs | accuracy (%): 98.1% -> 99.8% | Helped |
| Add GCL at late stage | iFormer-S · ImageNet-1K | accuracy (%): 83.4±0.40 -> 84.8±0.28 | Helped |
| Add GCL at late stage | iFormer-B · ImageNet-1K | accuracy (%): 84.6±0.45 -> 86.1±0.30 | Helped |
| Add GCL at late stage | ViT-B/16 · ImageNet-1K | accuracy (%): 74.3±0.51 -> 75.8±0.33 | Helped |
| Add GCL at late stage | ViG-B · ImageNet-1K | accuracy (%): 82.3±0.42 -> 84.0±0.30 | Helped |
| Use GCLs at all stages rather than selected late stage (ablation) | GCR · CIFAR-10 | accuracy (%): 94.07±2.14 -> 93.92±2.15 | Hurt |
| Use GCLs at all stages rather than selected late stage (ablation) | GCR · CIFAR-100 | accuracy (%): 74.74±5.73 -> 74.42±5.49 | Hurt |
| Use GCLs at all stages rather than selected early stage (ablation) | GCR · Tiny ImageNet | accuracy (%): 54.32±9.39 -> 54.13±9.49 | Hurt |
| Use adaptive layer weighting instead of fixed weighting (ablation) | GCR · CIFAR-10 and CIFAR-100 | accuracy gain | Helped |
Surprising findings
- Applying GCLs to all layers (Full) does not always give the best gains, suggesting naive aggregation can weaken regularization.
Limitations
- Gains on CIFAR-10 are modest and some configurations do not improve over baseline
- Theoretical bounds rely on loose or approximate steps (e.g., perturbation argument and Rademacher bound)
Experimental setup
Datasets: Kaggle cats vs. dogs CIFAR-10 CIFAR-100 Tiny ImageNet ImageNet-1K
Benchmarks: CIFAR-10 CIFAR-100 Tiny ImageNet ImageNet-1K Kaggle cats vs. dogs
Metrics: top-1 accuracy accuracy gain
Baselines: Baseline (no GCL) CNNs and transformers
Builds on: supervised contrastive learning graph Laplacian regularization center loss deeply-supervised auxiliary losses
Closest papers by meaning
- Exploring Consistency in Graph Representations: from Graph Kernels to Graph Neural Networks (NeurIPS 2024 · similarity 0.88)
- One Prompt Fits All: Universal Graph Adaptation for Pretrained Models (NeurIPS 2025 · similarity 0.88)
- A Unified Graph Language Model for Multi-Domain Multi-Task Graph Alignment Instruction Tuning (NeurIPS 2026 · similarity 0.88)
- UniGTE: Unified Graph–Text Encoding for Zero-Shot Generalization across Graph Tasks and Domains (NeurIPS 2025 · similarity 0.87)
- Multi-View Empowered Structural Graph Wordification for Language Models (AAAI 2025 · similarity 0.87)
- GraphTOP: Graph Topology-Oriented Prompting for Graph Neural Networks (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.