Research map / Large language models / Reasoning and chain of thought

Bridging On-Device and Cloud LLMs for Collaborative Reasoning: A Unified Methodology for Local Routing and Post-Training

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Routing, pruning and distillation for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes letting an on-device LLM decide internally when to offload queries to a cloud LLM, trained via RL post-training with hierarchical rewards, a group-level policy gradient, and adaptive prompt filtering (GAPG).

Problem

Deciding per query whether an on-device LLM should answer locally or offload to a cloud LLM under an offloading budget; existing external routers struggle to judge reasoning difficulty from the prompt alone and are trained separately from the on-device model.

Approach

Embed routing into RL post-training of the on-device model: a hierarchical reward (accuracy reward for local correct answers, smaller coordination reward for correct cloud-assisted answers) is optimized with a group-level policy gradient, plus adaptive prompt filtering that keeps locally solvable prompts and cloud-only-solvable prompts under a cloud-usage ratio to avoid policy collapse.

Key results

• On MATH-lighteval-tuned Qwen2.5-1.5B, the method reaches 66.7% average overall accuracy versus 60.8% for Task-Tuning&Router and 43.7% for Task-Tuning Only, under a 30% cloud-call budget.
• On Llama-3.2-3B, the method reaches 61.7% average overall accuracy versus 54.1% for Task-Tuning&Router, with 79.5% on MATH-lighteval.
• On Countdown with Qwen2.5-3B-Instruct, the method improves accuracy by about 30% relative to Task-Tuning Only and approaches Cloud LLM performance.

Surprising findings

• Collaboration-Aware Tuning (Dr. GRPO plus hierarchical rewards) underperforms the proposed method and on several benchmarks is below Task-Tuning&Naive Offloading in overall accuracy.
• Setting the coordination reward too high (alpha_c = 1.0) causes the model to collapse into always requesting help, capping accuracy near the 30% budget.

Limitations

• Restricted to reasoning tasks with verifiable correctness rewards; open-ended tasks would need quality-aware rewards.
• Hierarchical reward weights (alpha_a, alpha_c) are hyperparameters requiring tuning, and performance is sensitive to them.
• Prompt filtering can yield an empty set if pass@G of the on-device model is near zero.

Experimental setup

Datasets: Countdown MATH-lighteval

Benchmarks: MATH-lighteval MATH-500 AMC23 MinervaMath AGI-Eval-Math Countdown

Metrics: testing accuracy conditional local accuracy overall accuracy call-for-cloud ratio cloud invocations during training response length

Baselines: Task-Tuning Only (GRPO) Task-Tuning&Naive Offloading Task-Tuning&Router (DeBERTa-v3-large) Collaboration-Aware Tuning (Dr. GRPO + hierarchical rewards) AutoMix Router-R1 Cloud LLM (DeepSeek-R1)

Builds on: GRPO Dr. GRPO DeepSeek-R1

Closest papers by meaning

• HybridFlow: Resource-Adaptive Subtask Routing for Efficient Edge-Cloud LLM Inference (ICML 2026 · similarity 0.88)
• Cost-efficient Collaboration between On-device and Cloud Language Models (ICML 2025 · similarity 0.88)
• Intelligence per Watt: Measuring Intelligence Efficiency of Local AI (NeurIPS 2026 · similarity 0.88)
• Router-R1: Teaching LLMs Multi-Round Routing and Aggregation via Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
• Rectifying LLM Thought from Lens of Optimization (ICLR 2026 · similarity 0.86)
• Cloud-Device Collaborative Learning for Multimodal Large Language Models (CVPR 2024 · similarity 0.86)