Tether: Fine-tune BitNet models up to 13B parameters locally on phones and GPUs

What this is about

Stablecoin company Tether has presented a framework for fine-tuning Microsoft's language model BitNet b1.58. It is meant to run on any graphics card and on handy consumer devices. The announcement was published as a sponsored post on TechCrunch, so the numbers are vendor claims. For the full release, Tether points to Hugging Face.

Fine-tuning means retraining an already trained model on your own data, such as specialist texts. BitNet b1.58 is a model whose weights (the numerical values learned during training) take only three values: -1, 0 and 1. This is called ternary quantization, and it works out to roughly 1.58 bits per weight.

The problem: heavy resource demands

The post opens with AI's appetite for resources: electricity, cooling water and raw materials for chips. For context, it gives some examples. A Gemma 3 model with 1 billion parameters in half precision (FP16, 16 bits per weight) needs at least 2.2 GB of video memory (VRAM) for inference. That works on a laptop costing around 700 dollars with 8 GB of VRAM. Larger models would require a gaming PC for about 1,500 dollars with an RTX 5060 Ti (16 GB). According to the text, training and fine-tuning cost significantly more than pure inference, which means running a finished model.

The usual workarounds are FP16 instead of FP32 (half the memory, slightly less precision) or smaller models. Both still leave training and fine-tuning to multi-GPU servers with NVIDIA hardware and the CUDA ecosystem. CUDA is NVIDIA's programming platform for GPUs.

What Tether built

Microsoft's BitNet replaces costly floating-point multiplications with additions and subtractions using ternary values. That saves resources, but according to the post it suits CPUs better, since GPUs are built for floating-point arithmetic.

Tether has developed a new GPU backend based on Vulkan, a vendor-neutral graphics and compute interface. It is similar to the backend of Tether's QVAC Fabric LLM framework. It is meant to run BitNet on almost any graphics card, without NVIDIA or CUDA. Tether claims this is the first demonstration of BitNet efficiency on a GPU, with inference up to eight times faster than on CPUs.

The fine-tuning framework also uses an algorithm called dynamic tiling. It splits large matrices into smaller "tiles" whose size can be adjusted to fit into the fast GPU cache (SRAM). This is meant to improve speed and memory use, save energy and preserve precision across CPU and GPU.

Numbers according to Tether

The fine-tuning times refer to a biomedical dataset of about 300 documents (roughly 18,000 tokens). The framework is said to be able to fine-tune variants of up to 13 billion parameters on devices like the Samsung S25, Google Pixel 9 and iPhone 16. Tether calls this the first successful adaptation of a 13-billion-parameter model on an iPhone 16. The reported measurements, however, only cover models of up to 1 billion parameters; the text gives no runtimes for 13B.

The promised benefits

Tether cites cost savings, simpler development and privacy: data stays on the device and isn't uploaded to central servers. The company sees local AI as a way to bring "superintelligence" to billions of users.

Our take

Technically, it's interesting that BitNet fine-tuning runs on Vulkan regardless of vendor, and on smartphones too. That could make local, private model customization easier. The caveats: the text is sponsored advertising, and the benchmarks haven't been independently verified. Comparisons with FP16 models are of limited value because model quality differs, and the post says nothing about the answer quality of the BitNet models. The claim of 13B fine-tuning on the iPhone is also hard to judge without runtimes. It's also unclear what license the framework uses and how the weights are made available; the source gives no details on that.