Research map / Large language models / AI and society
Position: Require Frontier AI Labs To Release Small "Analog" Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Position paper arguing that frontier AI labs should be required to release small, openly accessible "analog" models distilled from their proprietary frontier models, to enable public safety and interpretability research without full disclosure.
Problem
Safety regulation of frontier AI models faces opposition over the safety-innovation tradeoff and cost of compliance; the paper seeks a middle-ground policy that improves safety oversight without burdening labs.
Approach
Mandate that any lab releasing a frontier model also publicly release a distilled analog model (0.5%-5% of frontier size, trained on the same data and objectives) after a 1-3 month lag, arguing safety interventions transfer from analogs to frontier models and that the compliance cost is roughly 0.1% of frontier training.
Key results
- Cost estimate for producing an 8B-parameter analog model is about $92,759, roughly 0.1% of a frontier training budget (estimate from AWS on-demand pricing).
- Summarized prior work reports steering vectors transferred from 0.5B-1B models to larger models with trigger rates collapsing from about 100% to under 5%.
- Reports that LLaMA-family uptake provides a natural experiment with the original LLaMA paper accruing about 15.8k citations (descriptive, not a controlled evaluation).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Port a steering vector learned on Qwen-0.5B using a two-layer auto-encoder | Qwen-1.5B and Llama3-3B sleeper-agent backdoors · Sleeper-agent backdoor removal | trigger rate: ≈100% to <5% | Helped |
| Transfer steering vectors from 1B or 3B Llama-3 models | Llama-3 8B · Seven steering directions | dose–response curve reproduction | Helped |
| Transfer an anti-toxicity vector from GPT-2 Large | GPT-2 XL · Toxicity | Perspective-API toxicity: 20–25 % margin | Helped |
| Use outputs from GPT-2 124M as supervision for GPT-4 fine-tuning | GPT-4 · MMLU, APPS, TruthfulQA | performance: over 90% | Helped |
| Use LoRA or knowledge distillation to compress large models | Trillion-parameter models · Capability preservation | capability loss: <2% | Helped |
| Map token embedding spaces across model sizes | 1B–70B checkpoints · Cross-scale representation alignment | Pearson correlation: r>0.9 | Helped |
Limitations
- Transferability of safety interventions from analog to frontier models is largely supported by cited prior work, not new experiments; emergent frontier-only behaviors may not appear in analogs.
- Cost estimates are first-order and exclude personnel costs beyond a doubling heuristic.
- Policy details (size caps, lag, enforcement) are proposals without empirical validation.
Experimental setup
Metrics: cost estimate (USD, GPU-hours)
Builds on: weak-to-strong generalization Platonic Representation Hypothesis LoRA knowledge distillation
Closest papers by meaning
- Position: AI Safety Must Embrace an Antifragile Perspective (ICML 2025 · similarity 0.86)
- AI Control: Improving Safety Despite Intentional Subversion (ICML 2024 · similarity 0.86)
- SafetyAnalyst: Interpretable, Transparent, and Steerable Safety Moderation for AI Behavior (ICML 2025 · similarity 0.86)
- Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress? (NeurIPS 2024 · similarity 0.86)
- Position: A Safe Harbor for AI Evaluation and Red Teaming (ICML 2024 · similarity 0.85)
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.