FP8, INT8, NVFP4 or GGUF: Which model format is right for your GPU

What this is about

Go to download a new image model like Krea 2, Z-Image, Qwen-Image or MOSS-Image from Hugging Face and you'll be greeted by a list of files: bf16, fp8_scaled, mxfp8, int8_convrot, nvfp4, plus GGUF variants from Q3 to Q8. Each suffix stands for a storage format: how many bits are used to store the model's numbers, and under which scheme. Smaller formats fit on smaller GPUs, sometimes run faster, and change the output image to a greater or lesser degree.

Vendors offer next to no documentation on which format works best on which card. But several community benchmarks are now available. We collected them and compared them side by side. Our thesis: bit count alone tells you little. What matters is whether your card can compute the format in hardware, and how sensitive the specific model is.

The benchmarks at a glance

One caveat up front: according to its own test kit, the big Reddit benchmark includes not a single RTX 40 card. The author is explicitly looking for Ada owners. Claims about Ada therefore rest mainly on the L40S measurement on Hugging Face and the RTX 4070 from Japan.

Which card can actually compute which format

A format always saves memory. It only gets faster if your GPU has matching compute units. Otherwise the numbers get unpacked (dequantized) back to 16 bits before every operation, which costs time. According to comfy-kitchen, the library ComfyUI uses for these formats, FP8 needs at least the Ada generation (SM 8.9), while NVFP4 and MXFP8 need Blackwell. Comfy itself lists only RTX 50 and Blackwell Pro cards for NVFP4 and requires PyTorch built with CUDA 13.0. Without that version, the Comfy blog says NVFP4 can be up to twice as slow as FP8.

INT8 ConvRot is the newcomer here. Classic INT8 (8-bit integers) was long considered unsuitable because individual outlier values blow out the coarse number scale. ConvRot comes from a December 2025 paper (arXiv 2512.03673). It "scrambles" values group by group using a Hadamard rotation, so outliers get spread across many positions. According to a Japanese analysis, the format has been built into ComfyUI since v0.27.0 (July 2026). The paper's authors report a 2.26x speedup with 4.05x less memory for W4A4 on FLUX.1-dev.

Speed and fidelity: what the numbers show

The cleanest comparison comes from the MOSS-Image measurement on Hugging Face: twelve prompts and seeds, 1024×1024 pixels, 25 steps. Alongside speed, it measured LPIPS, a metric for how different an image looks to the human eye compared with the BF16 image from the same seed (lower = closer to the original).

On Ampere and Ada, INT8 ConvRot is faster and closer to the original than FP8. Only on Blackwell does NVFP4 narrowly pull ahead on speed, and it deviates the most. That matches the Reddit benchmark: there, INT8 ConvRot was "the most faithful format in every model tested," and about twice as fast as BF16 on a 3090. NVFP4 was fastest on RTX 50 but drifted the most. Unsloth also measures a mean LPIPS of 0.064 for INT8 on Qwen-Image-2.1, versus 0.112 for FP8.

The Japanese measurements confirm this on mid-range cards (Krea 2, 768×768):

On top of that, system RAM usage for the Raw model dropped by around 14 GB. With RAM prices where they are, that's a real argument. On the RTX 5090, the second Japanese author measured Krea 2 Turbo (1024², 8 steps): INT8 ConvRot 2.56 s, NVFP4 3.29 s, FP8 scaled 3.91 s, BF16 4.84 s. So NVFP4 isn't automatically the fastest, even on Blackwell. On Z-Image Turbo, though, NVFP4 came out ahead there with 1.24 s versus 1.46 s for INT8.

Where it doesn't work

The counterexamples matter, because they show how much the result depends on the system:

• A100: In ComfyUI issue #14824, INT8 ConvRot ran at only 33 to 50 percent of FP8 speed. Comfy developer comfyanonymous explained that the A100's INT8 units are only about twice as fast as FP16, compared with roughly four times on RTX 30/40/50. The issue was closed without a fix. One user, however, got about 1.4x the speed with a fresh install (CUDA 13, no extra attention packages).
• T4: One user reported that ConvRot was "always slower than GGUF" for them. That was never resolved.
• AMD: Reports go both ways here too. On an RX 7900 XTX, INT8 was two to three times slower than FP8; on Strix Halo it was slightly faster.
• Mac: On an M1 Max, the INT8 version of Anima was slower than BF16 (47.2 s vs. 42.2 s), because Apple's MPS has no INT8 matrix math.
• Video models: With Wan 2.2, users on the RTX 5090 and 4060 Ti reported only a 5 to 10 percent gain over FP8, and said motion was slightly less smooth.

4-bit: same format, wildly different results

4-bit is where it's clearest that bit count says little. W4A4 means both the stored weights (W) and the intermediate values during computation, the activations (A), are rounded to 4 bits. With W4A8, activations stay at 8 bits. The comfy-quant-bench repo benchmarked both on an RTX 3090:

The activations make the difference. The mean per-layer error on HunyuanVideo was 0.0695 with W4A8 and 0.2136 with W4A4. The author's rule of thumb: above 0.21 it breaks, below 0.15 it works. But he stresses himself that this isn't a calibrated threshold. Even "Z-Image works" is disputed: the Japanese test found visibly degraded skin and a slightly altered face with INT4 on Z-Image Turbo. For very small models like Anima (2B), the first INT4 conversions produced no usable images at all.

With LTX 2.5, audio is also a factor: Lightricks' official INT8 is 1.9 times closer to the original than W4A8 for video, and 2.9 times closer for audio. On the 3090, another benchmark found W4A8 about 15 percent slower than INT8 on Krea 2, while saving 39 percent of memory. On older cards, 4-bit is mainly a memory fix.

Two traps: text encoders and turbo LoRAs

Text encoders (the language model that translates your prompt) can also be squeezed to 4 bits, for example Gemma 3 12B from 22.70 to 8.31 GiB. But according to comfy-quant-bench, stock ComfyUI unpacks them before computing. That saves memory, not time. With a special node that bypasses this, Gemma's prompt processing dropped from 2.354 to 0.539 seconds. One Wan user confirms that the INT8 text encoder didn't speed anything up for them.

Turbo LoRAs, small add-on modules that let a model finish in 4 to 8 steps instead of 20 to 50, don't play well with finished quants. If you only apply such a LoRA to a quantized model at load time, the Reddit benchmark says most of its effect gets rounded away. The recommendation: bake it in at BF16 first, then quantize.

What about GGUF?

GGUF, familiar from language models, is just a storage format in ComfyUI. Developer city96 writes that the weights are unpacked "on the fly" and computed with ComfyUI's regular code. Comfy developer Kijai therefore calls GGUF "an unnecessarily slow format for diffusion models" and estimates it at more than twice as slow on INT8-capable hardware. On fidelity, though, GGUF Q8 is strong: in one Reddit benchmark series, Q8 beat INT8 ConvRot on Qwen-Image 2512 and Z-Image Turbo, but without offloading it ran about as slowly as BF16. At Q4_K_M, quality on Qwen-Image 2512 collapsed to 13.78 dB, worse than FP8. And the MOSS-Image GGUF wouldn't load in ComfyUI at all. GGUF still makes sense with stable-diffusion.cpp and when memory is extremely tight.

What fits on which card

One important point first: according to the Reddit benchmark, nothing failed to run outright. If a model doesn't fit in VRAM, ComfyUI offloads parts to system RAM and pulls them back at every step. That's slow, but it works. The table below is our assessment based on the measurements. Where no measurement exists, we say so. File sizes are as given in the sources, some in GiB.

Our take

For most owners of RTX 20 through RTX 50 cards, INT8 ConvRot in current ComfyUI is the best default choice. It comes closest to the original and is faster than FP8 on Ampere and Ada. On a 12 GB 3060, that's the difference between ten minutes and three and a half minutes per Krea 2 image. FP8 isn't wrong on RTX 40, but it wasn't better in any of the available measurements. NVFP4 only pays off on Blackwell with CUDA 13, and even there only if you value speed or memory over image fidelity. 4-bit is a memory fix with risks: W4A8 has proven robust so far, while W4A4 has to be checked model by model.

Plenty remains open. The largest benchmark has no RTX 40 card, the error metrics measure deviation from the original rather than beauty, and A100, AMD, T4 and Mac show that drivers, PyTorch version and installation can flip the results. If you switch, compare a fixed prompt-and-seed pair in BF16 and the new format before you delete the big file.