NVIDIA’s Dynamo, the open-source serving layer that sits above vLLM, SGLang and TensorRT-LLM, shipped v1.6.0 on 22 October. The project’s README describes it as built in Rust for performance and Python for extensibility. The notes count 654 pull requests from 148 contributors, 58 of them outside NVIDIA.
Router and Kubernetes
The router gains a request-classifier hook that can defer a request or suggest a worker, and every request now flows through a policy queue ordered by deadline. Scoring and queue settings move into one --router-policy-config file, and the old flags are slated for removal in v1.7.0.
On Kubernetes, deployments that use the native Rust endpoint picker (EPP) can now run more than one replica and survive losing a pod. The legacy Go EPP still needs a single replica. The runtime now warns on sustained Tokio queue pressure and exports a count as dynamo_tokio_queue_overload_warnings_total.
Coding-agent traffic
Several changes target agent workloads:
- Billing header stripped by default. Dynamo now removes the
x-anthropic-billing-headerpreamble Claude Code adds, to save tokens and keep prompt caches stable. SetDYN_STRIP_ANTHROPIC_PREAMBLE=0or pass--no-strip-anthropic-preambleto keep it. - Headers kept. An
agent_headersmap preserves thex-claude-code-*andx-codex-*families. - Sessions grouped. An opt-in session prefix index sends a subagent’s requests to the same worker as its parent session, so they share cache.
- No guessing. Non-streaming
/v1/responsescalls no longer infer tool calls from plain assistant text.
Engines
Bundled engines move to SGLang v0.5.21, TensorRT-LLM v1.3.0rc29 and vLLM v0.31.0, so check your engine pins with the upgrade.
Why it matters
Coding agents send long, repetitive prompts with many subagents. Dynamo is tuning routing around that pattern, which is where cache hits and GPU cost are won or lost.
Update, 28 October 2026: the Rust KVBM crates skipped 1.6.0 and arrived at 1.6.1
If you looked for the KV block manager crates at 1.6.0, they were not there. The Dynamo v1.6.1 release notes explain why: kvbm-kernels, kvbm-logical and kvbm-physical lost their package description in v1.6.0, crates.io rejects a crate without one, and so those three and kvbm-engine, which depends on them, were never published for 1.6.0. We checked crates.io: all four now list 1.6.1 as their newest version, published on 27 October at 08:47 Israel time, and kvbm-engine goes straight from 1.5.1 to 1.6.1.
The same patch release moves the TLS roots compiled into the ai-dynamo-runtime and kvbm wheels to rustls-webpki 0.103.16 and webpki-roots 1.0.10, dropping the emSign ECC Root CA - G3. The notes mark that as breaking for endpoints whose certificates chain to that root.

The Campfire
CommentsThanks. Your comment is waiting for the desk. Once it is approved it appears here within about a minute.
Add a comment