Ignis is a small Rust and CUDA inference server that serves Qwen models to local coding agents through OpenAI-compatible routes. Version 0.9 adds speech recognition, and it runs on the CPU, “in host memory rather than VRAM”, so the GPU stays with the text or image model.
Install 0.9.2, not 0.9.0 or 0.9.1
First, the packaging problem. The v0.9.2 notes say the 0.9.0 and 0.9.1 tarball, zip and container images were built without the cuda feature. The server started, reported ready and answered every request with the CPU mock’s placeholder text, logging only a warning. 0.9.2 is the same code built with CUDA, and the release pipeline now refuses to package a server that would fall back to the mock. If you installed either earlier build, replace the binaries or image; models and config stay.
What 0.9 adds
From the v0.9.0 notes:
- Qwen3-ASR on the CPU. “An executor of ignis’s own, in Rust, runs it on any CPU with AVX2.” Three entries ship:
qwen3-asr-1.7b(the default, 1.5 GiB), a q8 variant andqwen3-asr-0.6b. It loads at first use, unloads after 300 seconds idle, and is never downloaded automatically. - OpenAI routes.
POST /v1/audio/transcriptionstakes WAV, FLAC, MP3 or AAC, and requests forwhisper-1reach the speech model. Live transcription uses the Realtime routeGET /v1/realtime?intent=transcription, with voice activity detection on “Silero VAD ported to Rust”. - Speaker labels. NVIDIA’s Nemotron 3 Diarization, ported to Rust on the CPU, can tag each item with a speaker when a session turns it on.
- Dictation and a Transcribe view in the built-in Playground, with exports to text, Markdown, SRT or VTT.
The project’s own numbers
Measured CPU-only by the author: the 1.7B model runs at 4.1 to 4.7 times real time on a 10-core desktop and about 3 times on 4 cores. Word error rate was 3.23% against 3.06% for Transformers FP32 on eleven Italian dictations, and 4.05% against 3.63% on 40 FLEURS clips. Diarization error was 3.57% on twenty synthetic Italian conversations but 30.2% on three real English meetings, “mostly missed speech”. Small samples, so treat them as a sanity check, not a benchmark.
Before you try it
The GPU kernels are compiled for SM120a (RTX 50-series Blackwell) only, and the engine needs a separate .ninfer model artifact. The Qwen3-ASR conversions on Hugging Face are Apache-2.0; the diarization weights carry NVIDIA’s own licence, so read it before commercial use. Ignis is a small project, with about 45 GitHub stars and nearly all commits from one account, which is worth weighing for anything beyond a home lab.
Update, 31 October 2026: 0.9.3 turns on vision for 16 GB RTX 50 cards
Ignis 0.9.3, released on 30 October, adds a --vision-residency host|device setting. Under the new default, host, the 27B model’s 296 MB vision tower stays in pinned host RAM and is streamed to the GPU through three 25 MB slots for each image, and the image embedding pool is host-mapped, so neither holds VRAM between images. With that, the rtx50-16gb profile now turns vision on, with a 4,096-token image envelope.
The project’s own numbers, measured on an RTX 5090 held to a 16 GB budget: with vision on, the KV cache pool stays at 7,956 pages, the same as text-only, against 7,437 and 7,384 pages under device. A 1,024-token image encode costs about 7 ms more (49 ms against 42), decode speed is unchanged and image embeddings are identical to the bit. Upgrade notes: a 32 GB card’s start now needs about 18.6 GB of host RAM instead of about 18, a host start that cannot pin the tower is refused rather than falling back, and anyone who wants the text-only load on a 16 GB card must pass --vision-enabled false.
Update, 4 November 2026: 0.9.6 adds a fine-tune that is not Apache-2.0, and every catalog row now names its license
Ignis 0.9.6, released on 4 November, puts Swift 1.5 in the built-in catalog. Swift 1.5 is UkisAI’s post-training of Qwen3.8-27B, shipped as two Ignis artifacts under one id: swift-1.5-27b, an NVFP4 build for 32 GB cards and larger (18.1 GiB), and swift-1.5-27b-16g, a trellis build for 16 GB cards (12.5 GiB) that a 16 GB card also loads when asked for swift-1.5-27b.
The release notes say plainly that “Its license is not Apache-2.0”. Both artifacts carry UkisAI’s Swift Open License v1.0. Its LICENSE sets a “Threshold” of “gross revenue of one million United States dollars (US$1,000,000) or more, measured over the most recently completed fiscal year of You together with every Legal Entity that controls, is controlled by, or is under common control with You.” Section 5 then says: “Any Commercial Use of the Work or a Derivative Work by a Legal Entity that exceeds the Threshold is not licensed under this License,” and that such an entity “may obtain a separate written license for Commercial Use from Licensor (the ‘Swift Enterprise License’).” The threshold “shall not apply to a Qualified Non-Profit Organization’s use of the Work or a Derivative Work for Non-Commercial or Research Purposes.” Section 6 says the license does not limit “any rights You have in the Base Model”, which stays under Apache-2.0, and Section 12 ends the license automatically on any breach. The LICENSE files in Ignis’s two redistribution repositories are identical to UkisAI’s.
The bigger change for anyone running Ignis is that licenses are now visible in the tool. Per the notes, model list gains a LICENSE column (license in --format json), model download prints the license before it fetches, the Server view links it on each row, and GET /v1/models/catalog returns license: {id, url} for every entry. “No step asks to accept it: whoever downloads is bound by it.” The project’s ADR 0078 also records that the built-in catalog “may ship weights whose license restricts commercial use”.
The project’s own measurements, against Swift’s BF16 weights on an RTX 5090: top-64 KLD of 0.05355 for swift-1.5-27b and 0.04255 for the 16 GB build, and DFlash2 drafting decoding 3.71x to 5.27x and 3.91x to 5.22x faster than no drafter for the two builds, across 8,192 to 253,440 tokens of context. The release also adds an NVFP4 converter for Qwen3.8-27B fine-tunes, and the official binaries are still compiled for RTX 50 and RTX PRO 6000 Blackwell cards only. If you script model download in a commercial setting, read the LICENSE column first: on this catalog, not every row is Apache-2.0 any more.
Update, 9 November 2026: 0.10 runs on RTX 30 and RTX 40 cards
Ignis 0.10.0, released at 00:33 UTC on 9 November, is the first release that is not Blackwell-only. Its Windows zip, Linux tarball and container image now carry kernels for compute capabilities 8.6 and 12.0.
- RTX 3090 and 3090 Ti are listed as supported on the 27B model’s 16 GB entries.
- RTX 4090 and the 16 GB RTX 40 cards (4060 Ti 16 GB, 4080) are “verified for correctness”. The project publishes no speed figures for RTX 30 or 40 cards yet and asks users for reports.
- The NVFP4 27B, Flash-Next and the image model still need Blackwell. On an older card, a request for a Blackwell-only entry is refused with
409 model_above_gpu_generation. - A new
rtx-24gbprofile gives the 27B’s 16 GB entry 1,048,576 tokens of context on eight lanes on a 3090 or 4090, and a new INT8 prefill path is the default on RTX 30 and 40.
If you held off because of the “check your card first” warning above, a 24 GB RTX 30 or 40 card is now on the list.

The Campfire
CommentsThanks. Your comment is waiting for the desk. Once it is approved it appears here within about a minute.
Add a comment