llama.cpp
The engine behind Ollama and LM Studio, ready to use on its own, with a built-in chat interface in your browser.
llama.cpp runs GGUF language models on your own machine, whether that's a CPU or an Nvidia, AMD or Apple GPU. A single command downloads a model from Hugging Face and starts a server with its own web chat interface and an OpenAI-compatible API. New models and features often show up here before they reach the apps built on top of it, and you get more control over the settings.
How to try it
Commands from the official guides – just copy them.
Linux
# Install llama.cpp with the official install script
curl -LsSf https://llama.app/install.sh | sh
# or via Homebrew
brew install llama.cpp
# Download a small test model and chat with it right in the terminal
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF
# Or start a server with a web chat interface (the address appears in the terminal, usually http://localhost:8080)
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUFmacOS
# Install llama.cpp with the official install script
curl -LsSf https://llama.app/install.sh | sh
# or via Homebrew
brew install llama.cpp
# Download a small test model and chat with it right in the terminal
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF
# Or start a server with a web chat interface (the address appears in the terminal, usually http://localhost:8080)
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUFWindows (PowerShell)
# Install llama.cpp in PowerShell with the official script
irm https://llama.app/install.ps1 | iex
# or via Winget
winget install llama.cpp
# Download a small test model and chat with it right in the terminal
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF
# Or start a server with a web chat interface (the address appears in the terminal, usually http://localhost:8080)
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUFCommands from the official guide. New in the Toolbox since Nov 13, 2026.
Ask the AI assistant
Questions about llama.cpp, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.
The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.
Comments
What do you think? Already tried it? Write it here, no sign-up needed.
Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.