Research map / Architectures and efficiency / RNNs and normalization
We Should Chart an Atlas of All the World's Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Position paper arguing for charting all public models into a unified "Model Atlas" graph of models and weight transformations, and proposing symmetry-avoiding ML methods (graph kNN, probing, within-component weight learning) to infer missing structure, with a preliminary HF-based charting method.
Problem
Public model repositories contain millions of models that are largely undocumented, so their capabilities, provenance, and lineage cannot be reliably determined, wasting compute and blocking reuse, provenance tracking, and discovery.
Approach
Represent the model population as a directed acyclic graph (Model Atlas) whose nodes are models and edges are weight transformations, and recover missing nodes, edges, and attributes by treating models as data while sidestepping parameter permutation symmetry, e.g., by staying within connected components with consistent neuron ordering, using probe-based functional features, and applying temporal
Key results
- On atlas recovery, the proposed method reaches 78.87% (Qwen), 80.44% (Llama), and 85.10% (SD) accuracy, versus 32.81%, 19.32%, and 50.51% for MoTHer.
- Hub-based majority-vote imputation improves license prediction accuracy from 0.49 to 0.85 and pipeline_tag from 0.60 to 0.79 over graph-average predictions.
- Over 60% of Hugging Face models lack model cards, and fewer than 30% specify their parent model; 99.73% of parent-child pairs have earlier parent timestamps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the proposed atlas charting method rather than MoTHer | Atlas recovery · Qwen2.5-0.5B CC | accuracy: 32.81 -> 78.87 | Helped |
| use the proposed atlas charting method rather than MoTHer | Atlas recovery · Llama-3.2-1B CC | accuracy: 19.32 -> 80.44 | Helped |
| use the proposed atlas charting method rather than MoTHer | Atlas recovery · Stable-Diffusion-2 CC | accuracy: 50.51 -> 85.10 | Helped |
| add greedy algorithm (ablation) | Atlas charting method · Qwen2.5-0.5B CC | accuracy: 77.34 -> 78.87 | Helped |
| add greedy algorithm (ablation) | Atlas charting method · Llama-3.2-1B CC | accuracy: 78.98 -> 80.44 | Helped |
| add quantization handling (ablation) | Atlas charting method · Qwen2.5-0.5B CC | accuracy: 70.57 -> 78.87 | Helped |
| add quantization handling (ablation) | Atlas charting method · Llama-3.2-1B CC | accuracy: 36.59 -> 80.44 | Helped |
| add quantization handling (ablation) | Atlas charting method · Stable-Diffusion-2 CC | accuracy: 84.85 -> 85.10 | Helped |
| add deduplication (ablation) | Atlas charting method · Qwen2.5-0.5B CC | accuracy: 67.85 -> 78.87 | Helped |
| add deduplication (ablation) | Atlas charting method · Llama-3.2-1B CC | accuracy: 75.28 -> 80.44 | Helped |
| add deduplication (ablation) | Atlas charting method · Stable-Diffusion-2 CC | accuracy: 88.76 -> 85.10 | Hurt |
| add temporal consistency (ablation) | Atlas charting method · Qwen2.5-0.5B CC | accuracy: 75.47 -> 78.87 | Helped |
Surprising findings
- Quantized models are leaves in 99.41% of cases on Hugging Face, and descendant downloads exceed the model's own downloads for 50% of non-leaf nodes.
- Weight-based MoTHer performs worse than the proposed structure-and-temporal method on in-the-wild charting, and Price's model performs near random on some components.
Limitations
- Charting relies on ground-truth base_model metadata for evaluation, and some components were manually inspected and fixed.
- Preliminary method assumes each connected component shares consistent neuron ordering and uses heuristics (quantization, temporal, duplicates) that may not generalize.
- Black-box proprietary models cannot yet be charted with weight-based methods.
Experimental setup
Datasets: Hugging Face hub-stats dataset Hugging Face model repository metadata
Benchmarks: Atlas recovery on Qwen2.5-0.5B, Llama-3.2-1B, and Stable-Diffusion-2 connected components TruthfulQA (0-shot, mc2) and HellaSwag (0-shot) metric imputation Hub-based attribute prediction (license, pipeline_tag, model_type, library_name, relation_type)
Metrics: accuracy (edge recovery) MSE MAE correlation attribute prediction accuracy
Baselines: Random Majority vote Price's model MoTHer
Builds on: MoTHer Price's model equivariant networks for weight space probing-based model representations
Closest papers by meaning
- Model Provenance Testing for Large Language Models (NeurIPS 2025 · similarity 0.83)
- ModelLens: Finding the Best for Your Task from Myriads of Models (NeurIPS 2026 · similarity 0.83)
- MGit: A Model Versioning and Management System (ICML 2024 · similarity 0.83)
- REEF: Representation Encoding Fingerprints for Large Language Models (ICLR 2025 · similarity 0.83)
- WorldModelBench: Judging Video Generation Models As World Models (NeurIPS 2025 · similarity 0.82)
- Witness Overlap: Directional Provenance Inside Open-Weight Model Families (NeurIPS 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.