Two PCs, one model: when a home AI cluster actually pays off

Microsoft now links Windows PCs over USB4, Apple links Macs over Thunderbolt 5, and hobbyists are building clusters from old data center cards. We lined up the published benchmarks side by side. Mostly, a cluster gets you more memory. It only gets faster with the right technique, and a single big machine is often the better choice.

Analysis Nov 21, 2026 10 min read Original analysis by our editors
Illustrative image, AI-generated

What this is about

Good open language models are growing faster than the memory in ordinary computers, and RAM is expensive right now. So one idea suggests itself: link two, three or four machines you already own and run one big model across all of them. There's a lot of new tooling for this kind of tinkering. On November 19, Microsoft showed RDMA over USB4 for Windows as a developer preview. Apple has offered the same over Thunderbolt 5 since macOS 26.2. On Reddit, home projects like LabHerd and a cluster of twelve used V100 cards are making the rounds.

We lined up the published benchmarks side by side. Our take: A cluster mostly buys you memory, and only rarely speed. Whether it gets faster depends almost entirely on how the model is split.

Two ways to split a model

With pipeline parallelism, each machine gets a block of model layers. Think of an assembly line: machine A computes the first 30 layers, hands the intermediate result to B, and B hands it to C. So for each token, only one machine is working while the others wait. On top of that, every handoff adds network latency. This is how llama.cpp works in its RPC mode over a regular network (TCP).

With tensor parallelism, all machines work on every layer at the same time, each on part of the matrices. That can be faster, but the machines have to sync up several times per token. That requires very low latency, and this is where RDMA (Remote Direct Memory Access) comes in. It lets one machine write directly into another machine's memory without going through the networking software stack. Over USB4, Microsoft measures round-trip times of 28 to 50 microseconds for small messages, roughly a quarter of what Ethernet (UDP) takes over the same cable. According to Geerling, RDMA on the Mac cuts latency from 300 to under 50 microseconds.

The benchmarks side by side

Mac (Jeff Geerling, four Mac Studio M3 Ultra, Thunderbolt 5): According to Gigazine, llama.cpp RPC running Qwen3-235B dropped from 20.4 tokens/s on one Mac to 15.2 tokens/s on four, a 25 percent loss. The exo software with RDMA and tensor parallelism, by contrast, rose from 19.5 to 31.9 tokens/s. With DeepSeek V3.1 (671 billion parameters), four Macs managed 14.6 tokens/s (llama.cpp) versus 32.5 tokens/s (exo). Kimi K2 Thinking, with a trillion parameters, ran at 28.3 tokens/s on exo. Geerling himself writes that llama.cpp's RPC approach does scale, but inefficiently, and performance drops with every machine you add. One important caveat: he didn't measure how fast long prompts are processed.

Windows (Microsoft, AMD, Intel): Microsoft's test setup also uses a llama.cpp fork whose RPC runs over RDMA instead of TCP, with tensor parallelism added. According to AMD, three AMD mini PCs (Ryzen AI Max+ 395, 128 GB each) reached 21.98 tokens/s on Qwen3-235B in 4-bit, and 117.98 tokens/s for prompt processing. Microsoft measured about 18.5 tokens/s in its own lab. Intel connected PCs with Core Ultra Series 3 and 64 GB each:

PCs Context Time to first token Output
3 1K 1.1 s 49.4 tokens/s
4 1K 1.2 s 54.2 tokens/s
3 2K 2.9 s 49.4 tokens/s
4 2K 3.2 s 51.7 tokens/s

So the fourth PC adds 9.7 or 4.7 percent of speed, and the wait for the first word actually gets longer. Microsoft doesn't give a single-PC number for any of the models. Xenospectrum also criticizes the lack of details on repeat runs, variance and output length.

Linux (Strix Halo, Level1Techs forum): There's no hardware RDMA over USB4 here yet. The software-based Soft-RoCE managed only about 4.3 Gbit/s, roughly half of what plain TCP achieved over the same Thunderbolt link (about 9 Gbit/s). The reason: the CPU is the bottleneck. On some devices, only one of two USB4 links came up at all. According to a summary, LTT Labs called a two-machine cluster of Ryzen AI Halo systems running llama.cpp RPC "disappointing."

Nvidia DGX Spark: A user on the Nvidia forum runs DeepSeek-V4-Flash on four Sparks with vLLM, tensor parallelism and RoCE over a 200 Gbit switch. He measures 49.4 to 54.4 tokens/s for a single user. Another user split Nemotron-3-Ultra-550B across two Sparks using llama.cpp layer splitting and got about 5 tokens/s. His verdict: two machines are slower per token than one, and you only do it because the model won't fit otherwise.

The key missing comparison: a single machine

Since Microsoft provides no single-machine baseline, we added community benchmarks. According to the strix-halo-llm-perf collection, a single Strix Halo with 128 GB runs GPT-OSS-120B at 53.4 tokens/s. Petronella Tech measures 41 to 45 tokens/s, depending on the software. That's about the speed of Intel's three-PC cluster. Qwen3-235B doesn't fit on a 128 GB machine in 4-bit. In the coarser Q3_K_M quantization (the model's numbers are stored even more coarsely), it runs there at 17.2 tokens/s. So what the three-device AMD cluster mostly buys you is a better quantization, with barely any extra speed.

It's a similar story on the Mac: a single Mac Studio with 512 GB can already hold Qwen3-235B on its own. Four Macs running exo make it 64 percent faster, but cost four times as much. According to MacRumors, Apple promises three times the speed of a single machine for four linked M5 Ultra systems. We found no independent measurements of that.

Home projects: real splitting or just load balancing?

Not every "cluster" tool actually splits a model:

  • LabHerd, according to its Reddit post, is a Windows front end for llama.cpp. One machine is the boss, the others are helpers, and each gets the piece of the model that fits its memory. You can mix AMD and Nvidia cards. That's real splitting, but based on the description it's built on llama.cpp RPC, so it's a pipeline. We found no benchmarks.
  • LanLM (github.com/sakthiveltofficial/lanlm, MIT license) can do both: split a model via llama.cpp RPC, or distribute requests across machines that each hold a full copy. The readme says it plainly: splitting is for fitting a model, not for making a small one faster. Real benchmarks are still missing.
  • Ollama Herd only distributes requests across several Ollama machines. Each one needs the complete model. That raises throughput for multiple users but doesn't let you run bigger models.

The other route is used data center cards. According to IT之家, a lawyer (u/TumbleweedNew6515) assembled twelve V100-SXM2 cards with 32 GB each for about $9,450. They're organized into three pipeline stages of four cards each running tensor parallelism, connected via NVLink and PCIe. On top runs DeepSeek-V4.1-Flash in 4-bit. A correction to many reports: According to his table, the 153 to 184 tokens/s apply to a single code stream. Regular text comes in at 83 to 95 tokens/s, and three parallel code requests combined reach 344 to 443. The system ingests a 128K document in 28 seconds; at 256K, it takes 58 seconds to the first token. The price: up to 2,100 watts and a lot of noise. According to Tom's Hardware, the machine shut itself down once with four simultaneous long chats. All numbers come from the builder himself.

With its "poor man's AI workstation" starting at €900, Golem promises models of up to 200 billion parameters on used hardware, and on one machine. We couldn't check the details behind the paywall.

Costs, cables and catches at a glance

The table compares different models and price points. It shows orders of magnitude and isn't a direct comparison (prices: EVO-X2 per Geizhals/heise price comparison, November 2026; Mac per Geerling, 2025, in US dollars).

Setup Model Tokens/s Approx. price Per token/s Cables / catches
1× Strix Halo 128 GB Qwen3-235B Q3_K_M 17.2 €4,200 ~€245 none
3× Strix Halo (USB4 RDMA) Qwen3-235B Q4_K_M 18.5–22 €12,600 + cables ~€575–680 3 cables, 2 ports per PC; Insider build, Secure Boot off, test signing
1× Strix Halo 128 GB GPT-OSS-120B 41–53 €4,200 ~€80–100 none
3–4× Intel Core Ultra 3, 64 GB GPT-OSS-120B 49.4–54.2 not stated – 3 or 6 cables; same as above
1× Mac Studio M3 Ultra 512 GB Qwen3-235B 20.4 $11,699 ~$575 none
4× Mac Studio (exo, RDMA) Qwen3-235B 31.9 ~$40,000 ~$1,250 6 cables at ~$70 each; enable RDMA in Recovery Mode, crashes under load
12× V100 (DIY) DeepSeek-V4.1-Flash 83–95 (text), 153–184 (code) $9,450 ~$100–115 (text) 2,100 W, loud, shutdowns

With a full mesh (every machine wired directly to every other), cabling grows fast: N−1 ports per machine, N(N−1)/2 cables. Three machines need three cables, four already need six, five need ten. Daisy-chaining slows things down significantly, according to Microsoft. Windows adds more hurdles: every machine has to run the same Insider build and the same drivers, the connection doesn't re-establish itself after a drop, and according to Xenospectrum the llama.cpp fork silently falls back to TCP. The only way to tell whether RDMA is actually active is a single line in the log. New memory prices don't make single machines cheaper either: according to ComputerBase, a Mac Studio M5 Ultra with 512 GB starts at €25,079, and according to Notebookcheck an RTX Spark desktop with 128 GB costs $7,499. And according to Thurrott, multiple RTX Spark desktops can't be linked anyway.

Our verdict

Based on the available numbers, a cluster only pays off if the model doesn't fit in the largest single machine you can afford. If it does fit, a single machine is almost always just as fast or faster, cheaper per token/s, and free of tinkering headaches. Up to about 120 billion parameters in 4-bit, a 128 GB machine will do (Strix Halo, DGX Spark, RTX Spark). A Mac Studio with 512 GB takes you up to roughly DeepSeek V3.1 in 4-bit (about 378 GB). A trillion-parameter model like Kimi K2, on the other hand, needs several Macs.

If you link machines you already own using pipeline tools like llama.cpp RPC, LabHerd or LanLM, expect more room and less speed. Only tensor parallelism with RDMA makes things faster, and even then the curve flattens out from the third or fourth machine. Windows RDMA is still a preview, with no available build and no single-machine comparison. Linux doesn't have fast RDMA over USB4 yet. The Mac is furthest along, but at prices where a single big machine costs half as much per token/s. For bargain hunters, single machines with MoE offloading and used data center cards remain interesting. But the fastest numbers from that camp are all self-reported by the builders.

Sources

#local ai#cluster#rdma#hardware#llama.cpp#comparison

Ask the AI assistant

Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.

Connecting …

The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.

Comments

What do you think? Already tried it? Write it here, no sign-up needed.

  1. Loading comments …

Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.