Research map / Label-efficient and robust learning / Recognition and long-tail
Demystifying Network Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an intrinsic evaluation framework (embedding geometry, CICFlowMeter metric alignment via CKA, and causal sensitivity tests) and applies it to four network foundation models across five datasets, finding anisotropy, weak context separation, and payload dependence.
Problem
Network foundation models are evaluated almost solely by downstream fine-tuning performance, which reveals little about what knowledge their hidden representations encode about network conditions and protocol behavior.
Approach
Evaluate frozen NFM embeddings intrinsically with three complementary analyses: anisotropy and mean cosine contribution for space utilization, CKA alignment with CICFlowMeter features, and perturbation/context-sensitivity tests (token replacement, NetReplica-emulated contexts), plus decorrelation to show anisotropy affects downstream F1.
Key results
- Mean cosine similarity of embeddings is 0.86 ± 0.09 across models, indicating significant anisotropy.
- Decorrelation of NetMamba embeddings raised F1 by +0.35 ± 0.02 on Crossmarket with a linear classifier.
- netFound had the highest average CKA alignment with CICFlowMeter features (0.143), while ET-BERT had the lowest (0.029).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply decorrelation to frozen NetMamba embeddings (ablation) | NetMamba · Crossmarket | ΔF1: +0.35 | Helped |
| apply decorrelation to frozen NetMamba embeddings (ablation) | NetMamba · CIC-IDS2017 | ΔF1: +0.112 | Helped |
| apply decorrelation to frozen NetMamba embeddings (ablation) | NetMamba · CIC-APT-IIoT24 | ΔF1: +0.033 | Helped |
| add BatchNorm to frozen NetMamba embeddings (ablation) | NetMamba · Crossmarket and CIC-APT-IIoT24 | ΔF1: +0.129; +0.029 | Helped |
| add BatchNorm to frozen NetMamba embeddings (ablation) | NetMamba · CIC-IDS2017 | ΔF1: -0.052 | Hurt |
| apply whitening to frozen NetMamba embeddings (ablation) | NetMamba · Crossmarket, CIC-IDS2017, and CIC-APT-IIoT24 | ΔF1: +0.228; +0.031; +0.008 | Helped |
| use netFound instead of ET-BERT for frozen embeddings | network traffic representations · CICFlowMeter features, average across five datasets | CKA: 0.029 -> 0.143 | Helped |
| perturb payload tokens | NFMs on CAIDA · CAIDA | cosine similarity between original and perturbed embeddings: YaTC: 0.18; ET-BERT: 0.48; netFound: 0.99; NetMamba: 0.62 | Mixed |
| perturb TCP SEQ/ACK fields | YaTC · CAIDA | cosine similarity: 0.87 -> 0.61 | Hurt |
| perturb TCP Window Size | YaTC · CAIDA | cosine similarity: 0.87 -> 0.67 | Hurt |
| use netFound embeddings to linearly probe network context | netFound · Synthetic datasets, all context changes | linear probe F1: 0.97 | Helped |
| change network context to AQM | netFound embeddings · Synthetic AQM dataset | linear probe F1: 0.60 -> 0.93 | Helped |
Surprising findings
- Payload perturbations shift embeddings substantially (YaTC 0.18, ET-BERT 0.48, NetMamba 0.62 cosine), despite payload being often encrypted in production.
- Models rarely separate high-level network context: for most model-dataset pairs the absolute cosine change was under 15% of the baseline gap.
Limitations
- Scope of exploration and metrics is non-exhaustive; initial attempt at intrinsic evaluation.
- Cosine similarity may give misleading insights for models trained with cosine-based losses.
- Public datasets may be flawed or biased in traffic patterns; synthetic context dataset is small (about 100 flows per context).
Experimental setup
Datasets: Android Crossmarket CIC-IDS2017 CIC-APT-IIoT24 CAIDA MAWI NetReplica-generated synthetic dataset
Benchmarks: Android Crossmarket CIC-IDS2017 CIC-APT-IIoT24 CAIDA MAWI NetReplica synthetic contexts
Metrics: anisotropy (mean cosine similarity) mean cosine contribution CKA similarity index cosine similarity change F1 silhouette score intrinsic dimension (TwoNN)
Baselines: YaTC ET-BERT netFound NetMamba
Builds on: ET-BERT netFound YaTC NetMamba NetReplica
Closest papers by meaning
- Foundation Models Secretly Understand Neural Network Weights: Enhancing Hypernetwork Architectures with Foundation Models (ICLR 2025 · similarity 0.85)
- Towards Effective Federated Graph Foundation Model via Mitigating Knowledge Entanglement (NeurIPS 2025 · similarity 0.84)
- Multi-Domain Graph Foundation Models: Robust Knowledge Transfer via Topology Alignment (ICML 2025 · similarity 0.84)
- Unearthing Skill-level Insights for Understanding Trade-offs of Foundation Models (ICLR 2025 · similarity 0.83)
- Specialized Foundation Models Struggle to Beat Supervised Baselines (ICLR 2025 · similarity 0.83)
- Exploring Representations and Interventions in Time Series Foundation Models (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.