A 284-billion-parameter model now runs on a 64 GB desktop with no GPU. The project, llama-modchip, fits DeepSeek V4 Flash into 57.61 GiB of memory and reports 9 to 15 tokens a second on a 16-core AMD chip, its README says. The headline is real. The speed is real, on the author's hardware. The question the project does not answer is whether the answers are any good at 1.71 bits per weight.
How 284 billion parameters fit in 64 GB
The trick is not magic. It is arithmetic, and it starts with the model itself. DeepSeek V4 Flash is a mixture-of-experts model with 284.33 billion total parameters, but only 13.17 billion of them activate for any given token: six of 256 routed experts, plus one shared expert, across 43 layers. A dense model of the same size would need to touch every parameter for every token. This one touches about 4.6 percent. The second trick is quantization. The project transcodes Unsloth's UD-IQ1_S build of the model, which occupies 76.87 GiB, down to 56.56 GiB at 1.71 bits per weight, a custom format it calls Fractal-64, while keeping the embeddings, the shared experts, and the attention machinery at higher precision. The whole model plus a 256K-token context cache peaks at 57.61 GiB of memory, which fits inside the roughly 62.5 GiB a 64 GB Linux machine actually has free. The third trick is kernels: patched AVX-512 routines the author says lift prefill by 77 percent and decode by 53 percent, plus a tiny speculative drafter that guesses the next tokens so the big model only has to verify them. The project, version 2.4.0, was created on October 29, 2026, and reached Hacker News' front page on November 8. It is a set of patches and tools for llama.cpp, not a product, and it requires its own patched binary: stock llama.cpp rejects the custom tensor format.
| Setup | Model file | Memory used | Decode speed | Measured by |
|---|---|---|---|---|
| llama-modchip, lossless Top-6 | 56.56 GiB (1.71 bpw) | 57.61 GiB | 9.1 to 9.9 tok/s | Repo author (self-reported) |
| llama-modchip, PVS drafting | 56.56 GiB (1.71 bpw) | 57.61 GiB | 11.5 to 14.7 tok/s | Repo author (self-reported) |
| ComputingForGeeks, 3-bit, 128 GB Xeon | 103 GB (3.06 bpw) | About 110 GB | 5.61 tok/s | Independent guide, Aug 2026 |
| Gist, 2-bit SSD streaming, 62 GiB | 81 GB (2-bit) | 62 GiB plus SSD | About 1.9 tok/s | Independent gist |
| vramglass, 2-bit, 128 GB unified | 97 GB (2-bit) | 128 GB unified | About 37 to 38 tok/s | Independent guide |
What 9 tokens a second buys
Nine tokens a second is slow by every standard that matters for chat. A 500-token reply takes about 55 seconds at that speed, and a 2,000-token answer takes over three and a half minutes. Nobody waits that long for a chatbot. But speed is not the only use for a model. At 9 to 15 tokens a second, DeepSeek V4 Flash becomes a batch machine: overnight reasoning jobs, offline document work, code review that runs while you sleep, and research on a machine that never touches the internet. The project's prefill speed, 44.4 tokens a second on the same 16-core chip, matters here too, because reading a long prompt is the part that happens before the slow generation starts. And the same project runs a smaller model far faster: NVIDIA's Nemotron 3.5 30B, with 3 billion active parameters, decodes at 25 to 60 tokens a second on the same CPU with no GPU, which is fast enough to talk to. The 284B number is the headline. The 30B number is the usable machine.
The quality question
Here is the part the README does not settle. The project calls its 284B build "lossless," but lossless means something narrow: the patched runtime produces the same logits as the unpatched runtime on the same checkpoint, a cosine similarity of 1.00000. That is a claim about the runtime, not about the model. The real quality question is what 1.71 bits per weight does to the model itself, and no independent benchmark of this build is published. The project's own 100-out-of-100 quality gate is self-measured, on its own tests, and the guides that measured this model independently used heavier builds: the 3-bit version at 103 GB, or the 2-bit version at 97 GB, both of which need more than 64 GB of RAM to hold comfortably. At 1.71 bits per weight, the experts that make up 96 percent of the model are stored at roughly one and a half bits each, and the honest statement is that nobody outside the project has published what that costs in accuracy. The speed table above is therefore a table of what is possible, not a recommendation. Anyone who downloads the 56.56 GiB file should test it on their own work before trusting it with anything that matters.
Why it matters
The direction is the story. Two years ago, a model this large needed a data center. Last year it needed a GPU cluster. Now it needs a desktop, a patched binary, and patience. Our small-AI scoreboard on October 24 showed what a 4 GB model can do, and the same logic applies at the other end of the scale: mixture-of-experts sparsity plus aggressive quantization keeps pushing the frontier onto hardware people already own. DeepSeek's model is open-weight, which is why any of this is possible at all, and the open-weight benchmark wars we mapped on October 16 are now being fought on laptops as well as clusters. But "runnable" and "good" are different claims, and the gap between them is measured in bits per weight. The project proved the first. The second is still open, and it is the test that decides whether a 284B model on a desktop is a tool or a demo.
By ByteHerald Staff. The llama-modchip project, version 2.4.0, created on October 29, 2026, reports that DeepSeek-V4-Flash, with 284.33 billion total parameters and 13.17 billion active per token, runs in 56.56 GiB as a 1.71-bits-per-weight build with peak memory of 57.61 GiB at 256K context on a 64 GB machine with no GPU, at 44.36 tokens a second of prefill and 9.06 to 9.95 tokens a second of decode, or 11.5 to 14.7 tokens a second with speculative drafting, on a 16-core AMD Ryzen 9 9955HX. Unsloth's UD-IQ1_S build of DeepSeek-V4-Flash occupies 76.87 GiB, and the project transcodes it to 56.56 GiB. An independent gist reports about 1.9 tokens a second of decode running the 2-bit, 81 GB build on a 62 GiB AMD Strix Halo machine via SSD expert streaming, with about 6 tokens a second of prefill. ComputingForGeeks measured 12.33 tokens a second of prefill and 5.61 tokens a second of decode for the 3-bit, 103 GB build on a 16-thread Xeon with 123 GiB of RAM and no GPU in August 2026. DeepSeek released DeepSeek-V4-Flash-0731 on July 31, 2026, with 43 layers, 256 routed experts, 6 firing per token, and a 1,048,576-token context window. ByteHerald reported on October 24, 2026 that a 4 GB model can do serious work on small hardware, and on October 16, 2026 that open-weight models from Meta, Xiaomi, and Mistral compete on capability, not only price.
Comments
Post a Comment