Research map / Large language models / Language, safety and interpretability

How Do Language Models Compose Functions?

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Pruning and circuits in LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that LLMs exhibit a compositionality gap on two-hop factual/lexical tasks and use either a compositional (computing f(x) en route) or direct mechanism, with the choice predicted by whether the task is a translation in embedding space.

Problem

Whether feedforward LLMs solve compositional two-hop tasks g(f(x)) using compositional mechanisms that compute the intermediate f(x), or through direct, idiomatic mechanisms, and what determines which is used.

Approach

Combines a compositionality-gap evaluation on many lexical two-hop tasks with logit lens and token-identity patchscope decoding of residual streams to classify compositional vs. direct processing, and links the choice to whether the task is approximated by a linear translation from input embedding to output unembedding space (r^2 = 0.65).

Key results

• Compositionality gap shrinks only modestly with scale for Llama 3 (1B to 405B: 72% to 39%) and OLMo 2 (1B to 32B: 74% to 49%).
• Reasoning model o4-mini reduces the gap from 49% to 18%, while DeepSeek-R1 reduces it from 41% to 31%.
• Across tasks, whether a task is represented as an embedding-to-unembedding translation correlates strongly with compositional processing for Llama 3 (3B) (r^2 = 0.65).

Surprising findings

• Compositionality gap persists even for the 405B Llama 3 model and shows no improvement between the 70B and instruction-tuned 405B models.
• Accuracy is only weakly correlated with the use of compositional mechanisms (r^2 = 0.22).

Limitations

• Analysis limited to single forward pass of feedforward LLMs and to specific task set; reasoning models only lightly tested.
• Logit lens and patchscope decoding do not guarantee causal use; some signals may reflect feature multiplicity.
• Some tasks (e.g. arithmetic) may be solved by algorithms not considered.

Experimental setup

Datasets: WikiData IMDb Non-Commercial Datasets Opus-MT translations antonym lists num2words

Benchmarks: compositional two-hop task suite (factual, lexical, arithmetic, color rotation, string manipulation)

Metrics: exact match accuracy compositionality gap reciprocal rank cosine similarity of translation reconstruction r^2 correlation causal effect of activation patching

Baselines: GPT-4o DeepSeek-V3 o4-mini DeepSeek-R1

Builds on: compositionality gap (Press et al.) logit lens token identity patchscope activation patching function vectors

Closest papers by meaning

• Compositional Reasoning in Language Models under Reinforcement Learning Post-Training (NeurIPS 2026 · similarity 0.87)
• AlgoTrace: Algorithmic Primitives and Compositional Geometry of Reasoning in Language Models (ICML 2026 · similarity 0.86)
• Task Generalization with Autoregressive Compositional Structure: Can Learning from $D$ Tasks Generalize to $D^T$ Tasks? (ICML 2025 · similarity 0.86)
• Vocabulary for Universal Approximation: A Linguistic Perspective of Mapping Compositions (ICML 2024 · similarity 0.86)
• When does compositional structure yield compositional generalization? A kernel theory. (ICLR 2025 · similarity 0.86)
• Limits of Deep Learning: Sequence Modeling through the Lens of Complexity Theory (ICLR 2025 · similarity 0.86)