Hugging Face Inference Providers now send LLM calls via OpenRouter, with retries and :fastest or :cheapest routing. Your own provider keys no longer work.
An SGLang remote code execution flaw let an attacker into NEAR AI's inference servers, about 100 accounts and 18 agents. Confidential mode was in scope too.
GPUStack before 2.2.0 hands admin to anything that looks like localhost, including traffic through a same-host proxy. CISA says it is exploited. Patch now.
Ferrite is a Rust LLM engine for DGX Spark with Rust GPU kernels. Since 0.1.0 it added Intel Arc Pro B70 and 16 GB RTX targets. Experimental, Apache-2.0.
Two more eighth-gen TPU systems, a TPU agent suite for coding tools in preview, GKE dynamic slicing GA and an Apache-2.0 C++ serving runtime. No prices yet.
Google says its Project Suncatcher prototype ran Gemma 4 on a data center class TPU in orbit. Next: radiation and heat tests, then two satellites in 2027.
A source-available Rust engine for Qwen GGUF models. Its author measures 113 tok/s decode vs 66.5 for tuned llama.cpp on an RTX 3060; prefill is level.
fal confirms a Bessemer-led Series F and a Sequoia-led Series E. Bloomberg reports $1.3B at $16B; CEO Burkay Gur says fal rate-limited users in summer.
FlashProve attaches a zero-knowledge proof to LLM output so a user can check which committed model produced it. Prototype only: 1,024 tokens, ~70-95x GPU cost.
Long prompts to GLM-5.3 on Workers AI could return text from other users' requests from 27 Aug to 9 Sep. Cloudflare and SGLang explain the bug and the fix.
TokenGate, from Stanford and Berkeley researchers, trains a gate that drops unneeded context tokens early, cutting prefill time and memory on small models.
TorchTPU, built with Meta's PyTorch team, is open source. Google adds TPU backends for vLLM, SGLang, Ray and VeRL, and device_map="tpu" in Transformers.
doubleAI's Maxwell optimizes programs while they run and says it formally proves each rewrite equivalent. It launched with a $200M Series A disclosure.
DeepInfer is Apache-2.0 and runs V4.1-Flash on NVIDIA SM100 or Huawei Ascend 950. This release is offline inference only, without KV cache or load balancing.