Research map / Reinforcement learning / RL from feedback

Dynamic Latent Routing

NeurIPS 2026 · Reinforcement learning › RL from feedback · approach: Scaling RL with verifiable rewards · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes General Dijkstra Search (GDS) with a proof that temporal concatenation of sub-policies recovers optimal goal-reaching policies in dynamic MDPs, and Dynamic Latent Routing (DLR), a single-stage LLM post-training method that learns discrete latent codes that steer the residual stream via search-select-update.

Problem

Discrete latent codes injected into LLMs during post-training disrupt pre-trained sequence structure and need multi-stage training, so they underperform SFT in low-data fine-tuning; there is also no theory for why temporal composition of sub-policies helps.

Approach

Replace the priority queue of GDS with a learned policy head that samples code sequences, selects the one maximizing conditional likelihood, and updates the policy head, codebook and LM jointly; codes are added as steering vectors to the residual stream at an injection layer rather than as extra tokens.

Key results

• DLR wins all 24 model-dataset cells vs SFT, with mean +6.6 pp gain over SFT (+7.8 pp on reasoning benchmarks).
• DLR reaches +18.8 pp on ScienceQA for Qwen3-8B and +10.2 pp on GSM8K for Llama3.2-1B.
• TokenAssorted collapses in low-data regime, e.g., 15.7% vs SFT 46.0% on GSM8K with Qwen3-0.6B.

Surprising findings

• TokenAssorted collapses to 15.7% vs SFT 46.0% on GSM8K with Qwen3-0.6B, showing inserted code tokens hurt low-data finetuning.
• Knocking out all codes in the 6-digit arithmetic model drops accuracy from 95.5% to 0.1%.

Limitations

• Low-data single-epoch regime evaluated; behavior with large budgets not the focus.
• Only QA benchmarks and small-scale arithmetic transformer evaluated.
• Theory on GDS applies to finite-horizon dynamic MDPs; link to LLM setting is a neural relaxation without formal guarantees.

Experimental setup

Datasets: GSM8K ScienceQA StrategyQA CSQA synthetic six-digit arithmetic

Benchmarks: GSM8K ScienceQA StrategyQA CSQA

Metrics: accuracy 95% bootstrap confidence intervals codebook cosine similarity codebook utilization topic purity accuracy drop under ablation

Baselines: SFT Pause Tokens TokenAssorted DLR (C=1)

Builds on: General Dijkstra Search Pause Tokens TokenAssorted representation engineering mutual-information skill discovery

Closest papers by meaning

• Memory-Enhanced Neural Solvers for Routing Problems (NeurIPS 2025 · similarity 0.88)
• Router-R1: Teaching LLMs Multi-Round Routing and Aggregation via Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
• LINC: Decoupling Local Consequence Scoring from Hidden Matching in Constructive Neural Routing (NeurIPS 2026 · similarity 0.87)
• Latent Plan Transformer for Trajectory Abstraction: Planning as Latent Space Inference (NeurIPS 2024 · similarity 0.87)
• PcLast: Discovering Plannable Continuous Latent States (ICML 2024 · similarity 0.86)
• Not Only Where, But When: Temporal Scheduling for RLVR (NeurIPS 2026 · similarity 0.85)