Research map / Architectures and efficiency / RNNs and normalization
Hedonic Neurons: A Mechanistic Mapping of Latent Coalitions in Transformer MLPs
Tanya Chowdhury, Atharva Nijasure, Yair Zick, James Allan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PAS (ablation-based) valuations instead of OCA in hedonic coalition discovery (ablation) | Hedonic coalitions (LLaMA-3.1) · CQTR OOD drop | OOD drop: 0.07 -> 0.11 | Improved |
| use hedonic coalitions instead of K-means or hierarchical clustering (ablation) | neuron coalition builder · OOD drop on CQTR | OOD drop: 0.02-0.03 -> 0.11-0.14 | Improved |
| use hedonic coalitions instead of clustering for feature alignment (ablation) | neuron coalition builder · MSLR IR heuristic alignment | alignment R^2: 0.11-0.18 -> 0.55-0.67 | Improved |
| use hedonic coalitions as macro-features for ridge regression (ablation) | coalition predictivity · DL-19/20 OOD, avg of 3 LLMs | ridge R^2: 0.15-0.21 -> 0.43/0.42/0.47 | Improved |
| use hedonic coalitions instead of spherical K-means for synergy (ablation) | neuron coalition builder · pairwise and ratio synergy, 3 LLMs x 3 tasks | synergy: +0.29 Pairwise, +0.49 Ratio | Improved |
| use random neuron subsets rather than clusters (ablation) | neuron coalition builder · pairwise synergy | pairwise synergy: 0.01 -> -0.23 for K-means on CQTR LLaMA vs Random | Hurt |
| track coalitions across consecutive layers 7 to 14 | cross-layer coalition tracking · CQTR/Mean-TF/L/RM, 3 LLMs | coalition fate percentage: vanish 60-75%, persist <12%, merges near 0% | Mixed |
Closest papers by meaning
- Scaling Large Language Model-based Multi-Agent Collaboration (ICLR 2025 · similarity 0.85)
- Attention as a Hypernetwork (ICLR 2025 · similarity 0.85)
- Talking Heads: Understanding Inter-Layer Communication in Transformer Language Models (NeurIPS 2024 · similarity 0.85)
- Latent Collaboration in Multi-Agent Systems (ICML 2026 · similarity 0.84)
- Bridging Critical Gaps in Convergent Learning: How Representational Alignment Evolves Across Layers, Training, and Distribution Shifts (NeurIPS 2025 · similarity 0.84)
- G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
