Research map / Large language models / Reasoning and chain of thought

Revisiting Parameter Server in LLM Post-Training

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient LLM serving · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes On-Demand Communication (ODC), which replaces FSDP's per-layer all-gather/reduce-scatter with point-to-point gather and scatter-accumulate so that the parameter-server pattern tolerates imbalanced LLM post-training workloads, and pairs it with minibatch-level load balancing.

Problem

Per-layer collective synchronization in FSDP creates barriers that leave devices idle when sequence lengths vary widely, as in LLM post-training (SFT and RL), limiting utilization and throughput.

Approach

Reframe FSDP as a decentralized parameter server by colocating server and worker roles and replacing collectives with on-demand RDMA point-to-point gather and scatter-accumulate, relaxing synchronization from once per layer to once per minibatch; this also enables minibatch-level load balancing (LB-Mini) that lets devices run different numbers of microbatches.

Key results

• ODC improves SFT throughput over FSDP collectives by up to 36% (14B, LongAlign, packed, minibatch size 4 with LB-Mini).
• On RL with GRPO, ODC achieves up to about 10% speedup over the collective baseline across 1.5B, 7B, and 14B models.
• Bubble rates drop sharply under ODC with LB-Mini; e.g., 7B LongAlign at minibatch size 4 falls from 30.31% (collective LB-Micro) to 7.19%.

Surprising findings

• Inter-node ODC primitives are significantly slower than NCCL collectives, though the paper argues long-sequence compute hides this overhead.
• On RL, ODC LB-Micro is slightly slower than collective LB-Micro for 14B at minibatch size 2 (-5%).

Limitations

• Inter-node communication overhead since point-to-point RDMA forgoes hierarchical collective optimizations and increases cross-node traffic.
• Speedup on RL is limited by verl constraints requiring equal sample counts per device.
• Keeps synchronous minibatch-boundary updates; asynchronous variants not explored.

Experimental setup

Datasets: LongAlign SWE-Smith AIME prompts

Benchmarks: LongAlign SWE-Smith AIME prompts

Metrics: samples per second per device speedup over collective bubble rate

Baselines: FSDP collective (all-gather/reduce-scatter) LocalSort LB-Micro verl Native packing

Builds on: FSDP ZeRO GRPO verl Karmarkar-Karp

Closest papers by meaning

• Rethinking Memory and Communication Costs for Efficient Data Parallel Training of Large Language Models (NeurIPS 2024 · similarity 0.88)
• Tackling the Data-Parallel Load Balancing Bottleneck in LLM Serving: Practical Online Routing at Scale (NeurIPS 2026 · similarity 0.88)
• SDP4Bit: Toward 4-bit Communication Quantization in Sharded Data Parallelism for LLM Training (NeurIPS 2024 · similarity 0.88)
• Efficient Pre-Training of LLMs via Topology-Aware Communication Alignment on More Than 9600 GPUs (NeurIPS 2025 · similarity 0.87)
• HALoS: Hierarchical Asynchronous Local SGD over Slow Networks for Geo-Distributed Large Language Model Training (ICML 2025 · similarity 0.87)
• ACCO: Accumulate While You Communicate for Communication-Overlapped Sharded LLM Training (NeurIPS 2025 · similarity 0.86)