Two-person lab trains open image generator from scratch for $21,000

What's going on

Linum, a research lab with just two people, has released a text-to-image model. It turns a text description into an image, and training it cost only about $21,000. The weights are available for download under the Apache 2.0 license, which also permits commercial use. The model was introduced on November 4, 2026, in a blog post and a "Show HN" post on Hacker News. Linum also published a Model Card on Hugging Face and a GitHub repository with code.

Linum's real focus is video generation. The goal is creative tools that make animation accessible to everyone. The image generator is a stepping stone toward the next video model, essentially a stress test for a new architecture. The developers say they were surprised themselves at how good the results look for such a small model, so they're releasing it as an experimental version.

The problem: video models are too expensive and too slow

According to Linum, today's video generators have two big weaknesses: they're too slow and too expensive, and they can't be controlled finely enough. Creative work means making something, tweaking it and improving it over and over. As long as every attempt costs a lot of time and money, video models are practically useless for storytelling. So with Linum v3, the main focus was cutting inference costs, meaning the cost of generating content with the finished model.

The bottleneck is attention, the mechanism a Transformer uses to relate every piece of an image to every other piece. Its cost grows quadratically with the number of tokens, the small units an image is broken into. Twice as many tokens means four times the compute. If you can represent images with fewer tokens, you save a lot on both training and inference.

How it works: pixels instead of a VAE

Compression is usually handled by a separate model called a Variational Autoencoder (VAE). It squeezes an image into a compact intermediate representation, the "latent space", and the actual diffusion model then works only in that compressed space. Linum doesn't think that's ideal. In the lab's experience, VAEs hit a ceiling on how much they can compress. And a VAE that reconstructs images better can even lead to worse results in the downstream diffusion model.

So Linum drops the VAE entirely. The new architecture, Pyramid-JiT (P-JiT), works directly in pixel space, on the pixels themselves. The diffusion model is meant to learn compression and image generation at the same time. Diffusion models create images by removing noise step by step, a bit like a picture slowly emerging from TV static. Linum says P-JiT needs 11x fewer training samples than Linum v2 to get good. It also cuts the token count four times more aggressively: a 512 × 512 image is represented with as many tokens as a 256 × 256 image used to be.

Two tricks make this work. First, Linum found a noise schedule that helps pixel models learn fine details. Second, there are intermediate outputs along the network. Other research such as REPA, RAE-v2 and Self-Flow suggests that the early layers of a Diffusion Transformer (DiT) only get weak learning signals. Because every layer in P-JiT predicts a version of the final image, you can check low-resolution variants early on and speed up training.

Architecture and training in detail

Text and image each first pass through two dedicated preprocessing blocks, called "refiners". They're then merged into a shared stream of 22 identical DiT blocks. The model understands text with the help of the frozen language model Qwen3.5-4B, which isn't changed during training. Linum reads its internal representations at three layers (7, 15 and 27). The image is split into 32 × 32 pixel patches and compressed into a 1024-dimensional space.

After block 5, an intermediate output aligns the internal representation with the vision model DINOv3-L. After blocks 10 and 16, two more outputs predict the clean image at a quarter and half resolution. Training started with high noise levels so the model would learn coarse structure. The range was then widened so it would learn fine details too.

For post-training, the fine-tuning after the main training run, Linum used a variant of Finite Difference Flow Optimization (FDFO), combined with the JEV-Image reward model and human raters. The developers call post-training for diffusion models a "very young discipline". For language models, it has driven big gains over the past 18 months, in coding for example, but for image models it's still in its infancy. The older DPO method overfits to particular aesthetic preferences. Linum suspects the difficulty comes from the fact that diffusion models, unlike language models, don't provide good likelihood estimates. The developers plan to share post-training details in the coming weeks.

Limitations

Linum is upfront about the weaknesses. The model can't render text in images, so no posters or infographics. Images with text, such as PDFs, book pages or slides, were deliberately removed from the training data because they don't matter for animation tools. The model also struggles with:

• full bodies (one example shows warped legs in a starting block),
• many objects, such as six cars or crowds,
• scenes where multiple people interact,
• world knowledge.

Sometimes these subjects come out fine. Linum says a larger model with around 7 billion parameters and more post-training should help.

The lab is also unusually transparent about how the texts were written: the developers wrote the blog themselves, while the Model Card and GitHub repository were generated automatically by Claude Opus 5.5 from internal experiment code.

What you need to try it

With 2 billion parameters for the image part plus 4 billion for the text encoder, the model is relatively small. The blog doesn't list hardware requirements for inference. Our rough estimate: in full bf16 precision, both parts together take up about 12 GB of VRAM (video memory). A card with 12 to 16 GB of VRAM should do. With a smaller version of the text encoder, or if you offload it to system RAM, cards with around 8 GB should work too. The source doesn't mention disk space, but plan for at least around 15 GB. Ready-made packages for ComfyUI or Ollama aren't mentioned. To get started, use the code from the GitHub repository with Python and PyTorch, ideally on an Nvidia GPU. Google Colab with a beefier GPU is a possible alternative; running on Apple Silicon or CPU only isn't documented.

The bigger picture

Linum isn't trying to beat industry heavyweights like FLUX, Midjourney or GPT-Image with this model, and it's too small to do so anyway. What stands out is the price: a usable text-to-image model trained entirely from scratch now apparently costs about as much in compute as a small car. That's within reach for universities, startups and hobbyist researchers. The Apache license and the very detailed documentation of the training settings make the project especially interesting for researchers who want to train their own models.

That said, all quality claims come from Linum itself. There are no independent benchmarks or comparisons, and the developers themselves call it an experimental release. Whether the VAE-free approach pays off for video, where the savings from fewer tokens matter most, is something Linum still has to prove.