Research map / Large language models / Reasoning and chain of thought

Collaborative LLM Numerical Reasoning with Local Data Protection

AAAI 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a local-remote collaboration framework for numerical document reasoning that rewrites queries to a shifted topic with preserved reasoning pattern and numbers obfuscated, then reconstructs answers by running the remote model's returned Python code with local values substituted back.

Problem

Low-capacity local models struggle with numerical reasoning over documents, but routing queries to remote LLMs leaks sensitive local data; existing protection methods (hints, examples) produce logically incoherent queries and local models cannot reconstruct correct answers from remote guidance.

Approach

A distilled local topic rewriter shifts the topic of context and question while preserving logical structure and numbers (then numbers are order-preservingly remapped); the remote model returns reusable Python code, and the local answer is recovered by executing that code with the inverse-mapped original numbers, avoiding local re-reasoning.

Key results

• On MultiHiertt, Phi-3-mini accuracy rises from 54.4% (single inference) to 80.1% with 3.7% leakage; on FinQA Llama-3.2-3B reaches 90.1% accuracy with 5.9% leakage.
• Compared with Hint and Example protection baselines, the method improves accuracy by 16.2%-43.6% and reduces leakage by 2.3%-44.6%.
• Ablation on FinQA: removing distillation drops accuracy from 90.1 to 42.2 (with tool) and removing tool drops it to 65.0.

Surprising findings

• Hint and Example collaboration methods yield lower accuracy than the local model alone on several settings, because logic-incoherent queries fail to elicit useful remote help.
• Vanilla DP-based text perturbation produces unreadable text unsuitable for reasoning tasks.

Limitations

• Remote LLM errors and annotation ambiguity account for a large share of failures (33.3% and 23.3% in error analysis).
• Leakage is judged by an LLM (GPT-4o-mini) with 96% agreement on a 200-sample manual check, so leakage measurements are approximate.
• Requires a remote model to return executable Python code and relies on retrieval of relevant context from the document.

Experimental setup

Datasets: FinQA MultiHiertt

Benchmarks: FinQA MultiHiertt

Metrics: accuracy normalized to remote-only local data leakage rate

Baselines: Single-inference Self-consistency Vanilla model cascading Hint Example InferDPT

Builds on: Program-of-Thought self-consistency model cascade knowledge distillation

Closest papers by meaning

• GSM-$\infty$: How Do your LLMs Behave over Infinitely Increasing Reasoning Complexity and Context Length? (ICML 2025 · similarity 0.85)
• Cost-efficient Collaboration between On-device and Cloud Language Models (ICML 2025 · similarity 0.84)
• ORACLE: Optimizing Reasoning Abilities of Large Language Models via Constraint-Led Synthetic Data Elicitation (AAAI 2026 · similarity 0.84)
• GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models (ICLR 2025 · similarity 0.84)
• ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning (ICML 2025 · similarity 0.84)
• Bridging On-Device and Cloud LLMs for Collaborative Reasoning: A Unified Methodology for Local Routing and Post-Training (ICML 2026 · similarity 0.84)