DeepSeek V4.1 Flash locally: Strata patch makes agent sessions start 20× faster

What this is about

A user in the r/LocalLLaMA community modified the local AI software Strata (MIT license, a desktop app and terminal tool for local models) so that DeepSeek V4.1 Flash starts agent sessions much faster on ordinary gaming hardware. The title of his post says "20× faster." The detailed result tables appear only as an image in the post and aren't available to us, so the figure is self-reported. The trigger was a remark by OpenAI CEO Sam Altman that it is "pretty bad" for everyone to have a chunk of scarce memory (DRAM) in their own computer that is used only about 5% of the time. The author wanted to show the opposite.

His machine: an AMD RX 9070 XT with 16 GB, a Ryzen 5800X3D, 64 GB of DDR4 RAM and a PCIe 3.0 NVMe SSD. On it, DeepSeek V4.1 Flash runs with 1 million tokens of context, fully offline. He uses the IST-DAS set at 1.75 bits per weight (bpw); according to the lab, it reaches 98.7% of the full model's average. The model's experts are spread across VRAM, RAM and SSD.

The problem: reading the prompt, not generating

Output at about 7 tokens per second was fine for the author. The bottleneck was reading in the prompt (prefill): every new session in OpenCode, a coding agent, re-read the entire system prompt including tool descriptions, about 37,000 tokens. About 100 seconds passed before the first response token.

Change 1: KV splice

Agents write session-specific text into the system prompt, such as the working directory, the date, the project file CLAUDE.md, or skill paths with random IDs. As a result, the prefix cache (a store for already computed prompt beginnings) misses on every new session.

The engine now compares a new prompt against all saved conversations. It cuts stale lines out of the saved state (the KV cache, the model's memory of tokens it has already read), shifts the saved lines, rotates the window rings and records a gap in the RoPE position encoding. Only the changed lines plus your own message are re-read. Changed lines are moved as whole lines, so the model never sees fragments.

Change 2: Resident-only routing

In a mixture-of-experts model, each token picks a few "experts." Even a short prompt touches nearly all experts in all layers, so a large part of the model was streamed from the SSD: 58 GB for a prompt of 1,200 tokens. Now tokens choose only while reading the prompt among the experts already in VRAM or RAM. Generation routes normally. That lets the system memory play to its strengths.

Limitations

• Tested only with HIP (gfx1201, AMD); CUDA, SYCL and Metal were neither built nor tested.
• The splice is approximate: tokens after a cut were computed with the removed text in context. With short hint lines the author saw no effect, but there is no formal evaluation.
• Resident routing changes answers slightly (substitute experts while reading the prompt). His heavy coding suite stayed at 12/13, but only one run each.
• Both changes are on by default in the branch; STRATA_DS4_REORDER=0 and STRATA_DS4_PROMPT_RESIDENT=0 turn them off. It also includes a one-line fix for a GPU page fault with a 37K-token prompt.

For OpenCode, the author recommends adding "interleaved": { "field": "reasoning_content" } to the model entry; otherwise OpenCode discards earlier reasoning and the cache misses more often. The patch is in the dsv41-kv-splice branch of the Strata fork. He asks for measurements, especially for CUDA, so the changes can be merged upstream (issues #4921 and #4936).

Background: Strata and DeepSeek V4.1 Flash

Strata runs on Windows, Linux and Mac and connects tools like Claude Code, Codex CLI or OpenCode through OpenAI- and Anthropic-compatible APIs. For DeepSeekV4.1-Flash-1017, the README lists sets at 1.75, 2.00 and 2.25 bpw, with downloads of 239 to 274 GB and roughly 245 to 280 GB of free space. Whatever doesn't fit in memory is read from the SSD. Supported hardware includes NVIDIA RTX 30/40/50, AMD Radeon AI PRO R9700 (Linux), Ryzen AI Max (Strix Halo) and Apple Silicon Macs with at least 64 GB, text only on the latter.

According to the README (2.00 bpw, Strata 0.2.5):

The model itself: according to SiliconANGLE, V4.1-Flash appeared on September 10, 2026, and the "1017" update a good five weeks later. DeepSeek calls it a "small update" that reduces task time and output length. On DeepSWE v1.1, it solved 77% of tasks in the company's own tests (previously 74.2%). According to TechTimes, it is an MoE model with 552 billion parameters, 8 billion active during prefill and 16 billion during decode, with a very small KV cache of 890 bytes per token. All benchmarks are vendor claims. The weights are on Hugging Face under the MIT license.

Try it: what you need

The model is huge: depending on the set, you need 245 to 280 GB of free storage, ideally on a fast NVMe SSD. Realistic setups are an NVIDIA RTX card from the 30 series up with 12 GB or more, roughly 46 to 64 GB of RAM, or a Mac with 64 GB or more. The patch described here has so far been built and tested only for AMD with HIP (gfx1201, e.g. RX 9070 XT), and you have to compile it yourself. Google Colab isn't a sensible option for this. Without the patch, Strata runs through the regular app.

Takeaway

The post shows that for agents, it's not generation but prompt reading that makes local use annoying, and that cache tricks can gain a lot. On the critical side: the numbers are self-measurements by a single user on one machine, the approximations haven't been formally evaluated, and resident routing changes the answers. The patch is an experimental branch, not an official release. Whether it works as well on CUDA or other platforms is an open question.