Llama-modchip runs a 284B DeepSeek on 64GB of RAM with no GPU

A llama.cpp fork posted to Hacker News on Sunday claims to run DeepSeek-V4-Flash — a 284-billion-parameter model — entirely in 64 GB of system RAM, with no GPU at all. The post passed 300 points within hours, and if the numbers hold up, it moves frontier-class models onto hardware a lot of people already own.

The project is called llama-modchip, and the trick is not one breakthrough but a stack of them: extreme quantization, mixture-of-experts sparsity, and a speculative-decoding drafter borrowed from chess engines. The result, according to the developer's own benchmarks, is a model that normally needs a server rack running on a 16-core desktop.

What it claims#

DeepSeek-V4-Flash has 284.33 billion parameters in total, but only 13.17 billion are active for any given token — it is a mixture-of-experts model with 256 routed experts, of which six fire per layer. That sparsity is what makes the whole thing plausible: most of the model sits idle while a small slice does the work.

The headline numbers, all from the project's README and measured on a 16-core Zen 5 desktop with 64 GB of DDR5, are:

• 56.56 GiB on disk for the full DeepSeek-V4-Flash checkpoint, at 1.71 bits per weight — down from 76.87 GiB for a standard low-bit build.
• 57.61 GiB peak RAM with a 256K-token context window, fitting inside the roughly 62.5 GiB a 64 GB Linux machine actually has free.
• 44.4 tokens per second on prefill and 9.1 tokens per second on generation, rising to 11.5–14.7 tokens per second with speculative decoding switched on.

The same fork also targets smaller models. Nvidia's Nemotron-3.5-30B-A3B, squeezed to 14.35 GiB, is reported at 25–60.4 tokens per second on CPU alone and up to 111.6 on a Threadripper workstation, and the README says 16 GB machines can run 30B models. Qwen3-30B is supported too.

How it fits: three tricks#

First, quantization. The fork introduces two custom formats: Fractal-64, a 2-bit format at 3.75 bits per weight using 64-weight blocks with a BF16 scale, and Fractal-MXFP4 at 4.25 bits per weight. The routed experts — 96 percent of the model's weights — are stored at around 1.5 bits each, with an analytical second-order technique the author calls SwiGLU-H2 matching the scales, which the README says cuts 20.31 GiB on its own. Sensitive layers are kept at higher precision so quality does not collapse.

Second, speculative decoding. A 1.64 MB n-gram drafter called Stockfish-PVS v2 — named for the chess engine's principal-variation search — guesses the next tokens so the big model only has to verify them. The README reports 96.4 percent acceptance, which is what lifts generation from 9.1 to as much as 14.7 tokens per second. A 13.1-millisecond snapshot restore for Claude Code sessions is thrown in for agentic coding workflows.

Third, the boring part that makes the rest work: cache-aware kernel tiling for AVX-512, register-stationary attention, and a 256K context that costs only a few hundred megabytes of KV cache. Memory bandwidth, not raw compute, is the bottleneck for CPU inference — and the whole design is built around that fact.

The catch: read the fine print#

None of these numbers has been independently verified. They come from the developer's own benchmarks, and the early Hacker News thread is mixed. One user reported the DeepSeek setup failing with a tensor shape error; another said Nemotron ran "just about the same as plain old llama.cpp on the CPU" in their test; a third reported a steady 12 to 14 tokens per second on a Ryzen 7 5700U with 64 GB of DDR4 — slower hardware, a result roughly in line with the claims.

There are practical limits, too. The custom Fractal-64 format uses a tensor type (GGML_TYPE_Q2_0, type 40) that stock llama.cpp, Ollama, and LM Studio will reject — you need the project's own prebuilt binary. The repository was created on October 29 and sits at version 2.4.0; it only surfaced widely today, when the Hacker News post took off. And 9 to 15 tokens per second is usable for chat and coding assistance, not blazing: it is roughly the speed of reading aloud, not the speed of a hosted API.

Why it matters#

The interesting part is not really the 284B number. It is the direction of travel: the floor for running a capable model locally keeps dropping. Two years ago a model of this class needed a data-center GPU; a year ago it needed a high-end card with offloaded experts; now the claim is a desktop you can buy off the shelf, with the model living entirely in RAM you can upgrade yourself.

That matters for more than hobbyists. Local inference means no API bill, no data leaving the machine, and no dependence on a provider's uptime or policy changes. It is the same reason people run a local LLM server with llama-cpp-python or follow our hands-on Ollama guide — only the ceiling just got much higher. DeepSeek's models have been a favorite for this kind of local work, including proactive coding agents that run on DeepSeek.

It also matters because RAM is getting expensive while GPUs are getting scarcer. A project that trades GPU silicon for system memory — and gets usable speed out of it — is aimed squarely at the hardware most people can actually buy.

What to try, and what to watch#

If you want to try it, the project ships prebuilt binaries and the quantized checkpoints on Hugging Face under the Synchopate account, plus a modchip command-line tool with chat, quantize, and benchmark subcommands. Bring a 64 GB machine, a fast SSD for the 57 GiB download, and patience: this is a research-grade fork, not a polished product.

What to watch is whether anyone reproduces the numbers. The README invites community benchmark submissions, and the Hacker News thread is already doing informal verification — the good kind and the skeptical kind. If independent runs land near 9 to 15 tokens per second with coherent output, llama-modchip becomes a genuine milestone for local AI. If they do not, it joins a long line of too-good-to-benchmark claims. Either way, we will know within days: the thread is still at the top of Hacker News, and the repo is public.

Sources: Hacker News, ‘Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU’ (November 8, 2026), here; synchopate/llama-modchip on GitHub, README v2.4.0, here; news.wrst.click, ‘Llama-modchip runs 284B DeepSeek-V4-Flash on CPU with 64 GB RAM’ (November 8, 2026), here; Claude Reads HN, ‘A 284B model runs on a 64GB desktop with no GPU’ (November 8, 2026), here. All performance figures are the developer's own, as stated in the project README; early user reports in the Hacker News thread are mixed. Cover and inline images are AI-generated illustrations.