Research map / Large language models / Language, safety and interpretability
Positional Cognitive Specialization: Where Do LLMs Learn to Comprehend and Speak Your Language?
Luis Frentzen Salim, Lun-Wei Ku, Hsing-Kuo Kenneth Pao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train only the 25% outermost layers (CogSym) instead of all layers | Llama 2 7B · Javanese/Sundanese downstream tasks | task performance: within 2-3% of full finetuning | No clear effect |
| train rear layers only instead of front layers only (ablation) | Llama 2 7B · word translation to Javanese/Sundanese | accuracy | Improved |
| train front layers only instead of rear layers only (ablation) | Llama 2 7B · word translation from Javanese/Sundanese | accuracy | Improved |
| train both front and rear regions together instead of one region with same per-region budget (ablation) | Llama 2 7B · word translation | accuracy | Improved |
| move trained layers inward from the model extremities (k=8) (ablation) | Llama 2 7B · word translation (Javanese) | accuracy | Hurt |
| select layers by FIM instead of CogSym positional heuristic (ablation) | Llama 2 7B · word translation (Javanese) | accuracy | No clear effect |
| select layers by LSN count or AlphaLoRA instead of CogSym (ablation) | Llama 2 7B · word translation (Javanese) | accuracy | Hurt |
| apply CogSym with LoRA instead of full finetuning | Llama 2 7B · word translation (Javanese) | accuracy | No clear effect |
| train regions sequentially (rear first) instead of simultaneously (ablation) | Llama 2 7B · translation toward unknown language | accuracy: almost 10% degradation | Hurt |
| train regions sequentially (front first) instead of simultaneously (ablation) | Llama 2 7B · translation toward unknown language | accuracy | No clear effect |
| train more than ten layers in one specialized region (ablation) | Llama 2 7B · region-specific tasks | accuracy | No clear effect |
Closest papers by meaning
- Exploring the LLM Journey from Cognition to Expression with Linear Representations (ICML 2024 · similarity 0.89)
- How do Large Language Models Handle Multilingualism? (NeurIPS 2024 · similarity 0.89)
- The Same but Different: Structural Similarities and Differences in Multilingual Language Modeling (ICLR 2025 · similarity 0.89)
- LLM Augmented LLMs: Expanding Capabilities through Composition (ICLR 2024 · similarity 0.89)
- The Rise and Down of Babel Tower: Investigating the Evolution Process of Multilingual Code Large Language Model (ICLR 2025 · similarity 0.88)
- Supervised Knowledge Makes Large Language Models Better In-context Learners (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
