llama.cpp and ggml reach v1.0.0, with semantic versioning for their C APIs
ggml-org tags llama.cpp and ggml v1.0.0 and promises major-version bumps for any breaking change to the public C API, which helps every app built on them.
Topic · 10 stories
10 stories tagged local-inference on AI Tamer, newest first.
ggml-org tags llama.cpp and ggml v1.0.0 and promises major-version bumps for any breaking change to the public C API, which helps every app built on them.
Velo Research open-sourced Kinetic, a Rust drop-in for llama.cpp's tools. Its big speedups are on file-rewrite tasks; plain generation is about level.
HOPE scores experts in pairs, not one by one. Its CLI writes a standard Hugging Face checkpoint. AWS reports 42.3 vs 41.8 at 50% pruning on Qwen3.5-35B-A3B.
MIT-licensed GPU Taskman v0.3.3, released 7 November 2026, reports per-process VRAM and estimates LLM weights and KV cache.
The Rust inference engine for AMD GPUs reports faster code edits on Strix Halo, native MTP on the 7900 XTX and Raon-1-mini-preview support.
The Rust local inference server now transcribes and labels speakers on the CPU behind OpenAI-style routes. Its 0.9.0 and 0.9.1 downloads shipped without CUDA.
Open Linux modules lift GeForce FP16 and BF16 tensor issue rates toward 2x on RTX 30 and 40 cards. Owners report 14 to 36% faster LLM prefill, not decode.
hipfire 0.4.3 adds a new mq4rs quant for Qwen3.6-35B-A3B with much faster prefill on gfx1201. 0.4.3.1 fixes a 27B MTP load bug and lifts decode about 48%.
Magnitude is an Apache-2.0 on-device inference engine for local agents: desktop app plus magnitude CLI, on-hardware kernel compile and tune, shared prefix cache, one-click agents or an OpenAI-compatible API.
A practical guide to choosing a local model runtime, understanding GGUF and memory needs, and deciding when a desktop setup is enough or a server makes sense.