AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

Outlier Matters: Efficient Long-to-Short Reasoning via Outlier-Guided Model Merging

Qiyuan Zhu, Dezhi Li, Lujun Li, Xiaoyu Qin, Wei Li, Hao Gu, Hua Xu, Sirui Han, Yike Guo

AAAI 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ORCA, an outlier-guided model merging framework that fuses base and reasoning LLMs to produce shorter reasoning chains (long-to-short reasoning).

Problem

Large reasoning language models produce extensive reasoning chains that incur high computational overhead; the goal is to shorten reasoning while preserving accuracy.

Approach

A plug-and-play model merging method that uses outlier activation patterns to guide how base models and reasoning models are fused, yielding concise reasoning without retraining.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add outlier-guided layer-wise allocation (ablation)Task Arithmetic (Qwen2.5-7B) · Six math benchmarks averageaccuracy (%): 53.6 -> 54.9Improved
outlier-guided instead of random allocation (ablation)Task Arithmetic (Qwen2.5-7B) · Six math benchmarks averageaccuracy (%): 52.7 -> 54.9Improved
increasing layer-wise allocation instead of uniform (ablation)Task Arithmetic (Qwen2.5-7B) · Six math benchmarks averageaccuracy (%): 53.6 -> 45.6Hurt
decreasing layer-wise allocation instead of uniform (ablation)Task Arithmetic (Qwen2.5-7B) · Six math benchmarks averageaccuracy (%): 53.6 -> 47.2Hurt
add magnitude-based ORCA probe (ablation)Task Arithmetic (Qwen2.5-7B) · Six math benchmarks averageaccuracy (%): 53.6 -> 54.2Improved
random probe selection (ablation)Task Arithmetic (Qwen2.5-7B) · Six math benchmarks averageaccuracy (%): 53.6 -> 53.9No clear effect
probe low-magnitude (minimum value) activations (ablation)Task Arithmetic (Qwen2.5-7B) · Six math benchmarks averageaccuracy (%): 53.6 -> 53.1Hurt
add adaptive alignment factor (ablation)four merging baselines (Qwen2.5-7B) · Six math benchmarks averageaccuracy (%): +1.2%Improved
add outlier-guided coefficient (ablation)four merging baselines (Qwen2.5-7B) · Six math benchmarks averageaccuracy (%): +1.5%Improved
add probe signals (ablation)four merging baselines (Qwen2.5-7B) · Six math benchmarks averageaccuracy (%): +0.6%Improved
add ORCA to Task ArithmeticQwen2.5-7B merge · Six math benchmarks averageaccuracy (%): 53.6 -> 55.7Improved
add ORCA to TIES-MergingQwen2.5-7B merge · Six math benchmarks averageaccuracy (%): 54.5 -> 56.3Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy