News and release notes

New: a local API other software can use

Sovereign Inference now exposes a documented HTTP API on your own machine, so other programs can use your local model.

• Two surfaces, one engine. Our own clean contract, plus an OpenAI compatible surface, so the openai library, LangChain and most existing tooling work by changing the base URL alone.
• Stateless by design. Nothing sent through the API is saved to your chats, added to your memory, or allowed to read either. No web search or cross chat recall fires.
• Text in, text out. The API exposes no file or folder access.
• Authenticated. A key is generated on first run and required by default.

Security

• The server now binds to your own machine only. It previously listened on every network interface, which put this machine's chats, memory and model on the local network with no authentication.
• The API key is never sent over HTTP, not even to Sovereign's own window, which reads it from your config file directly.
• Problem reports and logs are scrubbed. Anything key shaped is stripped before a report can be saved or emailed.

Better answers from more models

• Sovereign now uses each model's own embedded chat template rather than guessing the format from the filename.
• Fixed: h2o-danube models were prompted as Qwen, so they never saw a stop marker they recognised and rambled until they ran out of budget.
• Fixed: Dolphin 3.x models were prompted as Llama-3. Dolphin is ChatML whatever it was trained from.
• Fixed: Llama-3 prompts carried two begin of text markers.
• Fixed: any model name containing the letters "phi", such as Sophie or delphi, was treated as Phi-3.
• New safety net. Generation now stops at the end of turn markers of whichever template was actually used, so a mismatch degrades to a short wrong answer instead of a page of drivel.

Self diagnosis

• Settings, Support, Check for problems. Ten checks covering the runtime, folder permissions, a corrupt settings file, the models folder, the inference engine, the prompt template renderer, available memory against your largest model, the API key, network exposure and the port. Each one names the problem and what to do about it.
• Runs at startup too, printing only what is wrong, and is included in every problem report.
• The port check now tells you who is holding it rather than assuming.

GPU editions

• Four editions instead of one: CPU, Vulkan, CUDA 12 and CUDA 13, each installing alongside the others.
• The GPU editions now use the GPU out of the box. Offload previously defaulted to zero layers.
• Test which is faster times your loaded model on the processor and offloaded, and reports both. On an Intel Iris Xe we measured 8.7 tokens a second on the processor against 4.9 offloaded, on the same model. Integrated graphics are now called out in the interface.
• Mixture of Experts models can keep their expert weights on the processor, which is often both the only way a large model fits and the faster option.
• The System panel now reads the compute libraries actually present rather than claiming an edition.

Reliability

• Logs no longer grow forever. They roll over at 5 MB, keeping one previous copy.
• Crashes are now written down to their own file and included in problem reports, and the window tells you where it went.
• The API refuses oversized requests rather than allocating whatever a caller claims to be sending.

Interface

• The style and reasoning buttons now show which one is actually on. All four used to look equally active, which told you nothing.
• Reasoning is yours to control: off, on and hidden, or on and shown. It helps on multi step problems and costs time on look ups, and only you know which you are doing.
• Each conversation can have its own settings, saved with it, and a chat running on its own settings is marked rather than quietly differing.
• Models are checked before you commit to loading one. A model whose architecture the engine cannot run is marked straight away instead of costing another few minutes.
• A getting started screen on a new install, worked out entirely on your own machine. There is no recommended models list fetched from anywhere, because that would be exactly the phoning home we promise not to do.
• Copy the whole conversation, with Ctrl and Shift and C.
• Fixed: the prompt box jumping around when resized with the drag handle.

Known

• Ling 3.0 now works. It needs the bailingmoe3 architecture, which arrived in the engine build shipped with this release.
• Reasoning models that pre-open their thinking block are handled properly.
• Very new models depend on which engine build you have. If a model will not load, the message names the engine version, which is the number that answers whether a newer build would help.