Agents vs. CUDA: Much Faster at Reading Prompts, Barely Faster at Answering

What's going on

This week, two companies made nearly the same argument at nearly the same time. Anthropic is telling investors that its Claude Mythos model got Claude running on AMD's MI355X accelerator in about three days, a job that used to take six to twelve months. AMD published an experiment in which AI agents ported 51 current Hugging Face models to Strix Halo, its chip for laptops and mini PCs, in just a few days. The ports are supposedly faster than llama.cpp, the most widely used software for running models locally. The shared message: if agents can write the kernels, the small, heavily optimized compute programs that run directly on the chip, then Nvidia's CUDA software ecosystem stops being a moat.

We checked the numbers. One distinction is crucial here, and most coverage skips it. Prefill is the phase where the model reads in your prompt. The GPU can process many tokens at once, so speed depends on compute. Decode is the phase where the model writes its answer, one token at a time. For every token, all of the weights have to be read from memory once. That means the limit is memory bandwidth: how much data per second can move between memory and the chip. Decode speed is what you experience as "fast" or "slow."

What Anthropic actually claims

In the IPO roadshow video, compute lead Tom Brown says at around 20:42, according to an analysis by CellCog, that Claude was running on AMD "in just three days with virtually no human intervention" and had "ported or written almost 100 percent of the kernels." The footnote on the slide is narrower:

"The roughly three-day timeframe corresponds approximately to the time it took Claude Mythos Preview to bring prompt processing for one Anthropic model on AMD MI355X GPUs to a usable level of performance."

So this covers one model, and only prefill. Training isn't mentioned. Neither are tokens-per-second figures or a comparison with Nvidia hardware. According to the slide, the six to twelve months are a rule of thumb from earlier projects, not a benchmark result. CellCog and ExplainX report that neither "three days" nor "MI355X" appears in the prospectus itself. Wccftech calls it "premature" to declare CUDA dead and points to libraries, tooling and communication software such as NCCL, which is tied to Nvidia's NVLink interconnect.

Some background: Anthropic depends on both vendors. Nvidia committed up to $10 billion in November 2025, and AMD committed up to $5 billion in July 2026. Our take: heading into its IPO, Anthropic has a clear interest in looking independent of any single supplier. That doesn't make the claim false. But it's still a vendor claim with no data anyone can check.

The AMD experiment: open data, small contradictions

HRX Loom is a different story, because almost everything here can be checked. Loom is a language and compiler from AMD that generates programs for CPUs, GPUs and NPUs. It's still an early tech preview (version 0.1). Using the loom-skills skill pack, Claude Code, Codex or the Pi agent ports a model, benchmarks it against llama.cpp, optimizes it and writes up a report. The output is 51 model packages in the loom-fun organization on Hugging Face, along with a dashboard. Measurements were taken on six Strix Halo machines in AMD's DevLab plus one local device. The median run had 2 sub-agents working in parallel, with a peak of 16.

The sources don't even agree on the basics. The dashboard header says "4 days of work," while the methodology says "3 days" (November 6 to 9). Hardware Busters names AMD engineers Jeremy Fowers and Ben Vanik. The official Loom documentation, by contrast, says a "community contributor" ported the models over a weekend using a Claude subscription. Nowhere does it say how much humans reviewed or stepped in.

We ran the numbers on 14 model cards

Each card includes an info.json file with measurements against two llama.cpp variants: one for ROCm (AMD's counterpart to CUDA) and one for Vulkan. AMD's headline only compares against ROCm. For decode, we compare against whichever build is faster. That matters because the ROCm build falls apart at long context. An independent blog measured this on Strix Halo as well: with Qwen3.6-35B-A3B at 76,000 tokens of context, decode speed dropped 64 percent on ROCm and only 12 percent on Vulkan.

Our own calculation from the model cards. Loom compared with the better llama.cpp build; for prefill, compared with ROCm. According to AMD, differences of up to ±3% count as a tie.

Prefill: Here the lead is real and large. The median in our sample is about 1.6x, which fits AMD's figure of 1.5x across all 51 models. Even so, the agent-written kernels only reach 18 to 47 percent of the roofline, the hardware's theoretical ceiling. There's a lot of headroom left, and llama.cpp is apparently poorly optimized here.

Decode without tricks: For the seven models without speculative decoding, the median is +4 percent with short prompts and −1 percent at 32,000 tokens of context. In practice that's a tie. It also matches AMD's own plans: in the RFC for an HRX backend in llama.cpp, AMD engineer Stella Laurenzo expects "parity to 15% gains" for decode without multi-token prediction, and Phoronix reports up to 10 percent. The reason is physics. On Qwen2.5-32B, Loom hits 104 percent of the calculated bandwidth limit. Once memory is the bottleneck, a better kernel has nothing left to gain.

Decode with tricks: The big jumps only come with speculative decoding. A small add-on component guesses several tokens ahead, and the large model checks them all in one pass. The cards themselves say this comparison isn't like-for-like. Loom drafts 1 to 3 tokens ahead, while llama.cpp ran with up to 15, which can slow things down depending on the model. Independent measurements show that llama.cpp gains a lot from MTP on its own. According to sleepingrobots, Qwen3.8-27B (Q6_K, Vulkan) on Strix Halo went from 9.6 to 18.0 tokens per second. In another test, kernel patches alone added only +1 to +8 percent to decode, but much more to prompt processing.

The cards also openly list some methodological weaknesses. The machines changed between sessions. Loom was measured from the client, while llama.cpp was measured by its own server. Context sizes differed. Several values exceed the estimated roofline.

How solid is "correct"?

All 14 cards pass the "golden" and "compare" tests, which compare tokens and probabilities against llama.cpp on the same GGUF file. In other words, the yardstick is llama.cpp itself, not an independent full-precision reference. The hobbyist port inferzero for Intel iGPUs is stricter. It compares against a 32-bit CPU implementation using KL divergence, which measures how far two probability distributions differ. The full-context test (needle_full) wasn't rerun for the current package on 12 of the 14 cards. Earlier builds had passed it. Qwen3.8 Flash Next needs a development build whose installation instructions expire on November 22.

Research in this area also calls for caution. In the KernelBench-Verified paper, the best model, GPT-5.5, reaches only 0.88x PyTorch's speed under strict checking. Under the usual checking rules, it was 1.43x. SpecBench finds apparent gains with no real improvement in 73.8 percent of KernelBench evaluations. That's reward hacking: an agent games the test instead of solving the task. One example is a ReLU kernel that simply detects the test size and reports a 374x speedup. AMD's dataset paper AMDKernelVault reports that its trained Qwen3-8B gets 34 percent of HIP kernels right on the first try. On Intel GPUs, Xe-Forge averages 1.17x over PyTorch eager.

The other fronts

The SGLang consortium, announced on November 12, brings Nvidia, AMD, Intel, Google, Meta, Cloudflare, Modal and RadixArk together. Technical decisions still rest with the SGLang maintainers, though. And Nvidia is hardly losing out. Nvidia executive Ian Buck says better software helps developers get "more performance out of Nvidia compute." Intel separately announced a partnership with RadixArk to optimize SGLang for Intel hardware.

Google reports a 4.09x speedup across 17 reference TPU kernels from its /teamwork agent system, and 2.99x across 7 hand-optimized vLLM kernels. The much-cited "Rust kernels" are something else entirely: 800,000 lines of Rust for the Zircon operating-system microkernel and a Rust port of the AV1 decoder libgav1 that ended up faster than the C++ original. None of that has anything to do with GPUs.

Where CUDA still holds up

In August, SemiAnalysis used its AgentX benchmark to measure real agent workloads with up to a million tokens of context. With Qwen3.5 on SGLang, Nvidia was more than 20x ahead. Context-parallel attention, which spreads very long contexts across many GPUs, isn't supported in vLLM on AMD. On top of that come training, multi-chip communication (NCCL), mature libraries like cuDNN and running all of this at scale. None of this week's announcements mention any of these points.

The bottom line

Our take: what's cheap to port today is inference for individual models on a single chip, especially prefill. That's where llama.cpp on AMD is apparently weak, and where agents clearly deliver. If you own a Strix Halo, that's good news: long documents and agent sessions get processed noticeably faster. When it comes to writing the answer, memory bandwidth sets the limit, and clever kernels hardly change that. The bigger gains there come from speculative decoding, which llama.cpp supports too.

By Anthropic's own footnote, its "three days" are a prefill result for one model, with no published data. The CUDA moat is getting shallower where the work is clearly scoped and can be checked automatically. For training, multi-GPU deployments and reliability at scale, it's still there. The main open questions: who will maintain agent-written kernels over the long term, and how can you reliably rule out reward hacking in production systems?