Research map / Large language models / Reasoning and chain of thought

HybridFlow: Resource-Adaptive Subtask Routing for Efficient Edge-Cloud LLM Inference

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Efficient LLM serving · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes HybridFlow, which decomposes a query into a dependency DAG of subtasks and routes each ready subtask between an edge SLM (Llama3.2-3B) and a cloud LLM (GPT-4.1) using a learned utility predictor and an online budget-adaptive threshold.

Problem

Edge-cloud LLM collaboration usually routes whole queries or fixed-granularity steps by estimated difficulty, missing subtask-level parallelism and failing to adapt to time-varying latency and API budgets.

Approach

Combine an edge-side planner that builds a validated subtask DAG with dependency-triggered parallel execution, and route each subtask via an MLP utility predictor (trained on profiled benefit-cost targets) plus a dual-variable adaptive threshold that rises as cloud cost and latency are consumed.

Key results

• HybridFlow averages 55.34% accuracy across four benchmarks versus 46.50% for DoT and 38.70% for HybridLLM, with CoT on GPT-4.1 highest at 58.99%.
• HybridFlow reports average latency 17.48 s and average API cost 0.0088 per query, the lowest among edge-cloud baselines (DoT 18.32 s / 0.009; HybridLLM 24.45 s / 0.0128).
• On GPQA, HybridFlow reaches 53.33% accuracy while offloading about 40% of subtasks, versus 57.28% for all-cloud CoT at 0.0185 API cost.

Surprising findings

• Disabling DAG parallelism (HybridFlow-Chain) reduces accuracy and utility only modestly relative to full HybridFlow, and HybridFlow's latency gain over DoT is small (4.6%).
• On MMLU-Pro, HybridFlow's latency (11.85 s) is higher than DoT's (11.00 s), contrary to the claim that parallelism always reduces latency.

Limitations

• Offline utility targets depend on counterfactual edge/cloud profiling of every subtask, which is costly (about 31,836 s for 2,000 queries) and assumes the planner is not re-run conditioned on upstream outputs.
• Planner validity is only 76-78% without repair; about 9-10% of queries fall back to chain execution.
• The utility formulation and LinUCB calibration are described loosely; the paper reports results only on one edge-cloud pair for main experiments and one swapped pair for generality.

Experimental setup

Datasets: MMLU-Pro Math500 s1k GPQA AIME24 LiveBench-Reasoning

Benchmarks: GPQA MMLU-Pro AIME24 LiveBench-Reasoning

Metrics: accuracy end-to-end latency (s) cloud API cost ($) normalized cost utility

Baselines: CoT SoT PASTA HybridLLM DoT Direct Prompt Random routing Fixed threshold routing

Builds on: chain-of-thought Skeleton-of-Thought HybridLLM Explain-Analyze-Generate prompting knapsack / Lagrangian relaxation

Closest papers by meaning

• Bridging On-Device and Cloud LLMs for Collaborative Reasoning: A Unified Methodology for Local Routing and Post-Training (ICML 2026 · similarity 0.88)
• SpecEdge: Scalable Edge-Assisted Serving Framework for Interactive LLMs (NeurIPS 2025 · similarity 0.87)
• Intelligence per Watt: Measuring Intelligence Efficiency of Local AI (NeurIPS 2026 · similarity 0.85)
• Cost-efficient Collaboration between On-device and Cloud Language Models (ICML 2025 · similarity 0.85)
• TokenRouter: Efficient Serving System for Token-Level LLM Routing (NeurIPS 2026 · similarity 0.85)
• MAPS: Memory-Aware Predictive Scheduling Framework for Large Language Model Serving (ICML 2026 · similarity 0.85)