Ubuntu's Inference Manager puts all your local AI models under one roof

What's going on

Canonical, the company behind the Ubuntu Linux distribution, used its in-house Ubuntu Summit to show off a new tool: the Inference Manager. It lets you install and manage AI models directly on your own computer and make them available to other programs. Inference is the moment a trained model actually does its job, like answering a question or turning speech into text. According to heise, the tool is available now in the Snap Store, Ubuntu's official software source.

The Inference Manager is part of a broader AI strategy Canonical has laid out over the past few months. The core idea: AI in Ubuntu should run locally, build on open models and always be optional.

How the Inference Manager works

The tool gives you an inference command. You install it with sudo snap install inference or graphically through the Ubuntu App Center. After that, you set up a model with something like inference install smollm2. SmolLM2 is a very small language model that runs even on modest hardware.

Models ship as inference snaps. Snap is Ubuntu's own package format: a snap bundles a program with everything it needs to run and keeps it sealed off from the rest of the system. For AI models, the package holds more than the model itself. It also includes the matching runtime and, where available, acceleration settings tuned for your hardware. Startup Fortune reports that Canonical already offered optimized packages for models such as DeepSeek R1 and Qwen 2.5 VL on selected chip architectures during earlier betas. The goal is to save you from piecing together drivers, runtimes, quantized models (shrunk by storing numbers at lower precision) and hardware-specific settings yourself.

The real selling point of the Inference Manager is the unified interface. Every installed model can be reached through an HTTP interface on TCP port 8400. Canonical says it's compatible with the OpenAI API, the format apps typically use to talk to ChatGPT models. In practice, that means plenty of existing apps built for OpenAI's cloud can simply be pointed at your own machine. Desktop chat assistants, coding agents in your editor and dictation tools can all reach their models the same way, with no internet connection required.

First real-world example: dictation with Myna

Myna shows what this looks like day to day. The speech-to-text app, software that turns spoken words into text, arrived with the current Ubuntu release, 26.10 “Stonking Stingray”. According to heise, Myna offers smooth dictation on the desktop and types the result into any text field. It runs on the same inference snaps the Inference Manager now handles.

At the Summit, Canonical demoed the concept with a Carrera slot-car track where AI models controlled the cars. In a web app, visitors could pick different models as “drivers” that responded to voice commands like “go faster”.

The bigger strategy: implicit and explicit AI

Jon Seager, VP of Engineering at Canonical, described the roadmap behind the tool in a blog post, which Linux Journal and Developer Tech have summarized. Canonical splits AI features into two kinds:

• Implicit AI: Existing system features get quietly better with AI in the background, and the way you use them stays the same. Examples include speech recognition and text-to-speech, which mainly help with accessibility.
• Explicit AI: New, visible workflows, such as agentic ones. These are AI agents that take actions on your behalf, like writing documents or applications or automating tasks.

According to Developer Tech, Seager's concrete examples include troubleshooting a Wi-Fi connection and setting up an open-source software forge (a platform for managing code projects) with security and TLS encryption already configured. Admins should benefit too: site reliability engineers, the people who keep servers running, could have AI comb through log files during outages, dig into root causes or automate planned maintenance under strict controls.

These agents are supposed to follow the same safeguards as production systems: access controls, audit trails of every action, tightly scoped permissions and a clear line between just observing and actually acting. The question isn't only whether you can trust agents, Seager says, but whether they can work within the controls you already have.

Canonical is also keeping an eye on newer models that support tool calling. That lets a model call external programs or APIs, search the web or access files, as long as you allow it.

Openness, licensing and privacy

Canonical stresses that every AI feature will be opt-in, meaning nothing turns on until you explicitly agree, and you can uninstall it at any time. Since the features ship as modular snaps, that's easy to do technically. According to Linux Journal, local inference is meant to be the default, with cloud services used only when you deliberately choose them. Developer Tech reports that inference snaps follow the same confinement rules as other snaps, which is meant to limit what the models can reach on your machine and in your data.

Canonical favors open-weight models, whose weights you can download freely, along with open-source “harnesses”, the software that wraps and drives a model. But it doesn't want to rely on weight availability alone. It also plans to weigh each model's license terms. Seager is clear:

“Ubuntu is not becoming an AI product.”

Canonical is holding back internally as well. Employees won't be judged on how much they use AI. Seager is quoted as saying he'll keep measuring people by how well they deliver. Low-quality AI-generated contributions to open-source projects were never acceptable, he adds.

Limits: hardware and timing

Seager admits hardware is still the bottleneck. Small models run on everyday machines, but for many tasks they can't yet match large models. Canonical is watching consumer chips with stronger AI accelerators and wants to look at power draw as well as speed. Local accelerators could cut the energy needed for inference.

There's no firm timeline for individual features. They're due to land over the coming year, once Canonical considers them mature. The company plans to expand its work on inference snaps with newer models and optimized builds for more chip platforms. According to Startup Fortune, Ubuntu already supports hardware from Nvidia, AMD, Arm, Intel and others.

Our take

The idea isn't new: tools like Ollama and LM Studio have offered local models with an OpenAI-compatible API for a while. The difference is the integration into the operating system. If Ubuntu itself provides one central, standard address for local AI, apps can rely on it instead of each shipping their own models. Myna is an early example. The hardware-optimized packages could also spare newcomers the hassle of setting up drivers and runtimes.

What's still unclear is how big the model catalog in the Snap Store will get, how well the optimizations actually work across different hardware and how the Inference Manager stacks up against established tools. There are no independent tests yet, and the privacy and confinement claims come from Canonical itself. Startup Fortune also points out a risk: AI features that feel pushy, opaque or half-baked would quickly erode trust in a community that cares deeply about control. With its strict opt-in approach and focus on local, open models, Canonical is deliberately avoiding that trap, which sets it clearly apart from cloud-centric approaches like Windows.