A two-person research lab trained a text-to-image model from scratch for $21,038 in compute and published the receipt. Linum's Pyramid-JiT run is a pressure test for its next video model, and the interesting part is not the price alone. It is the architecture trick that made the price possible, and the honest list of what the model still cannot do.
The receipt
Linum published the full cost breakdown on its field notes blog, and the numbers are unusually complete. Pretraining cost $20,348, for 8,139 H100-hours at $2.50 per H100-hour, covering 610 million samples at 512 pixels and then 50 million at 1024 pixels, on 64 H100 chips. Post-training cost $690, for 276 H100-hours, using a variant of Finite Difference Flow Optimization with JEV-Image and human-in-the-loop judges. The total is $21,038 for 8,415 H100-hours, restarts included. By ByteHerald's arithmetic, post-training is 3.3 percent of the bill and pretraining is 96.7 percent, and the whole run saw 660 million samples.
| Stage | Cost | Compute | What happened |
|---|---|---|---|
| Pretraining | $20,348 | 8,139 H100-hours | 610M samples at 512px, then 50M at 1024px, on 64 H100s |
| Post-training | $690 | 276 H100-hours | FDFO variant with JEV-Image and human-in-the-loop judges |
| Total | $21,038 | 8,415 H100-hours at $2.50 each | 660M samples, restarts included |
The model itself is a 2 billion parameter Pyramid-JiT, or P-JiT, paired with a frozen 4 billion parameter Qwen3.5-4B text encoder, and Linum released the code and weights under the Apache 2.0 license. The lab describes the release as a research artifact rather than a full model release, and the field notes post carries no publication date, so the run is dated here by its appearance on Hacker News and in syndicated coverage in early November 2026.
The trick: throw away the compressor
Most image models compress pictures before they generate them. A separate model, usually a variational autoencoder, squeezes each image into a smaller latent space, and the diffusion model works in that space. Linum's argument is that the compressor is the bottleneck. It has an empirical ceiling on how much it can compress, and improving it at reconstruction can make generation worse downstream.
P-JiT drops the compressor entirely and works directly on pixels. Each token covers a 32 by 32 patch of the image, where most models use 16 by 16, as in FLUX-2. A 512 pixel image now costs the same number of tokens a 256 pixel image did before, a fourfold cut in what the network has to compare, and because attention cost grows with the square of the context window, that cut is the whole game. The model also predicts the image at several resolutions along the trunk of its diffusion transformer, so every block is, in effect, predicting a version of the final image, which gives the early blocks a stronger gradient signal. Linum reports that P-JiT reaches its previous model's quality on 11.3 times fewer training samples.
The quality evidence is Linum's own measurement, and it should be read as such. On FD-DINOv2, a distribution distance score where lower is better, measured on 30,000 generations at 256 pixels against a fixed reference set, P-JiT scored 27.9, against 39.1 for the lab's earlier JiT-DDT architecture and 79.9 for Linum v2. The real-vs-real floor, the same measurement run on 30,000 more real images instead of generations, is 25.4, so P-JiT sits close to the best score the measurement itself can give.
What $21,038 does not buy
Linum states the limits plainly, and they matter more than the headline number. The model was never trained on images of rendered text, no infographics, PDFs, pages of books, or slides, because the lab considers that data useless for its goal of animation tools. As a result it cannot write text or generate posters, full stop. It is a small model, a 2 billion parameter image network with a 4 billion parameter text encoder, and it struggles with full bodies, many entities such as six cars or a crowd, multi-person interactions, and world knowledge. Linum says a larger model, around 7 billion parameters, with further post-training would likely mitigate those weaknesses, which is a plan rather than a result.
There is also no head-to-head comparison against other image models in the post. The gallery shows 249 samples and the lab's own validation loss curves, but the quality claim rests on Linum's judgment for now. That is not a flaw in the work, it is the correct way to read it: a $21,038 research artifact from a two-person lab, not a frontier model with an independent benchmark.
Why a cheap image model matters for video
The run is a stepping stone, and Linum says so directly. Generative video models almost always start as generative image models, because a model has to learn nouns, people, places, things, before it can learn verbs, actions and movement. The image run is a pressure test of the architecture on the way to Linum v3, a video model, and the lab's stated goal for v3 is a tenfold improvement in speed and cost. The scale of the problem is visible in the lab's own numbers: its previous model, Linum v2, spent 110,000 tokens on a single 720p five-second clip. If the 32 by 32 patch trick holds up when the model has to handle motion, video training gets cheaper in the same way image training just did. If it does not, the $21,038 stays an image-model story.
The cost ladder for image models keeps falling, and this run is the newest rung on it. Our explainer on how image generation actually works and what it costs puts the $21,038 figure in context, from the diffusion machinery to the price of a single image at each tier. Linum's run also belongs to the same small-model economics as the 4 billion parameter models that run on a laptop: a model does not have to be frontier-scale to be useful, and a two-person lab can now afford to train one from scratch. The question the next field notes post has to answer is whether the trick survives the jump from still images to moving ones.
By ByteHerald Staff. Linum, in its field notes post "Training a Text-to-Image Model for $21K," reported a total compute cost of $21,038 for its Pyramid-JiT text-to-image model, comprising $20,348 for pretraining on 8,139 H100-hours and $690 for post-training on 276 H100-hours, at $2.50 per H100-hour, with 660 million samples seen and the code and weights released under the Apache 2.0 license. Linum, in its earlier field notes post "Pyramid-JiT: Low-Res Drafts, High-Res Images," reported that P-JiT reaches Linum v2's FD-DINOv2 score on 11.3 times fewer training samples, with scores of 27.9 for P-JiT, 39.1 for JiT-DDT, and 79.9 for Linum v2 against a real-vs-real floor of 25.4, measured on 30,000 generations at 256 pixels. A syndicated summary published in November 2026 confirmed the cost figures and noted the post contains no head-to-head comparison against other image models.
Comments
Post a Comment