Research map / Large language models / AI and society
Position: We Need An Algorithmic Understanding of Generative AI
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Position paper arguing for an algorithmic understanding of LLMs, proposing the AlgEval framework (algorithmic primitives and composition, analyzed via attention, representations, and circuits) with a small graph-navigation case study on Llama-3.1 models.
Problem
Research on generative AI emphasizes scale and performance, leaving little systematic understanding of which algorithms LLMs learn and use to solve problems.
Approach
Proposes AlgEval, a research program that identifies algorithmic primitives and their compositions in LLMs, combining top-down algorithmic hypotheses with bottom-up circuit, attention and representation analysis, illustrated on a tree-graph navigation task.
Key results
- Final token attends significantly more to rooms on the correct path than the incorrect path (b = 0.33, t(2015) = 4.51, p < .001), with significant correct-path attention in 14 of 32 layers for Llama-3.1-8B.
- Layer-wise matching of LLM representation steps to BFS/DFS rollouts yields mean proportion of correct matching steps of 0.18 (BFS) and 0.24 (DFS), so no classical search trajectory matches well.
- Llama-3.1-70B shows the same patterns with more pronounced separation between correct and incorrect trajectories and no difference in search strategy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Scaling model from Llama-3.1-8B to 70B | LLM graph navigation search strategy · Graph navigation (tree graph n=7) | search strategy match to BFS/DFS | No clear effect |
| Layer-wise representation sequence compared to BFS/DFS rollouts | Llama-3.1-8B hidden states · Graph navigation (tree graph n=7) | mean proportion of correct matching steps: 0.18 BFS, 0.24 DFS | Hurt |
Surprising findings
- Layer-by-layer representational similarity stays above 0.95, changing smoothly rather than in discrete algorithmic steps.
- Internal representation sequences do not closely match BFS or DFS rollouts, suggesting policy-dependent rather than classical search.
Limitations
- Case study is a single small two-step tree graph with one prompt format, limiting generality.
- Position paper; the framework's primitives and composition methods are proposed rather than validated empirically.
Experimental setup
Datasets: hand-constructed tree-graph navigation prompts (n=7 nodes)
Benchmarks: tree-graph navigation case study
Metrics: linear mixed-effects model on attention paired t-tests per layer t-SNE representation separation representational similarity (Procrustes, CKA) edit-distance match to BFS/DFS rollouts
Baselines: BFS hypothesis DFS hypothesis
Builds on: mechanistic interpretability Marr's levels of analysis RASP neural algorithmic reasoning GRPO
Closest papers by meaning
- Does GPT Really Get It? A Hierarchical Scale to Quantify Human and AI’s Understanding of Algorithms (AAAI 2025 · similarity 0.89)
- LLM Circuit Analyses Are Consistent Across Training and Scale (NeurIPS 2024 · similarity 0.87)
- Understanding the Challenges in Iterative Generative Optimization with LLMs (NeurIPS 2026 · similarity 0.86)
- Large Language Models Think Too Fast To Explore Effectively (NeurIPS 2025 · similarity 0.86)
- Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence (ICML 2025 · similarity 0.86)
- AlgoTrace: Algorithmic Primitives and Compositional Geometry of Reasoning in Language Models (ICML 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.