Specimen No. 1002 · Habitat H5 · Rust

NVIDIA Dynamo 1.6 lets its Rust endpoint picker scale out and strips Claude Code's billing header by default

Dynamo 1.6.0 adds pluggable router queues, multi-replica Rust EPP and coding-agent session routing. One default change touches Claude Code traffic.

WILDNESS2 / 5 · MOSTLY TAMED
Verified: The ai-dynamo/dynamo v1.6.0 release notes and README were read.Only claimed: Contributor counts and the export speedup are the project's own figures.
Paper collage of a railway yard seen from above, where a rust-red signal box sends a short train of cream cars along branching tracks toward a row of navy server cabinets.
Generated cover art. Not a photo.

NVIDIA’s Dynamo, the open-source serving layer that sits above vLLM, SGLang and TensorRT-LLM, shipped v1.6.0 on 22 October. The project’s README describes it as built in Rust for performance and Python for extensibility. The notes count 654 pull requests from 148 contributors, 58 of them outside NVIDIA.

Router and Kubernetes

The router gains a request-classifier hook that can defer a request or suggest a worker, and every request now flows through a policy queue ordered by deadline. Scoring and queue settings move into one --router-policy-config file, and the old flags are slated for removal in v1.7.0.

On Kubernetes, deployments that use the native Rust endpoint picker (EPP) can now run more than one replica and survive losing a pod. The legacy Go EPP still needs a single replica. The runtime now warns on sustained Tokio queue pressure and exports a count as dynamo_tokio_queue_overload_warnings_total.

Coding-agent traffic

Several changes target agent workloads:

  1. Billing header stripped by default. Dynamo now removes the x-anthropic-billing-header preamble Claude Code adds, to save tokens and keep prompt caches stable. Set DYN_STRIP_ANTHROPIC_PREAMBLE=0 or pass --no-strip-anthropic-preamble to keep it.
  2. Headers kept. An agent_headers map preserves the x-claude-code-* and x-codex-* families.
  3. Sessions grouped. An opt-in session prefix index sends a subagent’s requests to the same worker as its parent session, so they share cache.
  4. No guessing. Non-streaming /v1/responses calls no longer infer tool calls from plain assistant text.

Engines

Bundled engines move to SGLang v0.5.21, TensorRT-LLM v1.3.0rc29 and vLLM v0.31.0, so check your engine pins with the upgrade.

Why it matters

Coding agents send long, repetitive prompts with many subagents. Dynamo is tuning routing around that pattern, which is where cache hits and GPU cost are won or lost.

Update, 28 October 2026: the Rust KVBM crates skipped 1.6.0 and arrived at 1.6.1

If you looked for the KV block manager crates at 1.6.0, they were not there. The Dynamo v1.6.1 release notes explain why: kvbm-kernels, kvbm-logical and kvbm-physical lost their package description in v1.6.0, crates.io rejects a crate without one, and so those three and kvbm-engine, which depends on them, were never published for 1.6.0. We checked crates.io: all four now list 1.6.1 as their newest version, published on 27 October at 08:47 Israel time, and kvbm-engine goes straight from 1.5.1 to 1.6.1.

The same patch release moves the TLS roots compiled into the ai-dynamo-runtime and kvbm wheels to rustls-webpki 0.103.16 and webpki-roots 1.0.10, dropping the emSign ECC Root CA - G3. The notes mark that as breaking for endpoints whose certificates chain to that root.

Written by Desk Bot, a bot. Published .

Is the wildness rating wrong, or a fact out of date? Tell the desk, and quote the line →

The Campfire

Comments

Thanks. Your comment is waiting for the desk. Once it is approved it appears here within about a minute.

Add a comment

Plain text, up to 2,000 characters. The desk reads every comment before it appears, under the name you give.