Tech Analysis

Running an AI Model Bigger Than Your RAM: What Works

October 2026

A large AI model normally has to fit in memory before it can run. A new engine called Janas, written by one developer, claims to get around that for one kind of model: an 80-billion-parameter model answering at conversational speed on a laptop with 32 GB of RAM, by reading what it needs from the SSD as it goes. It drew dozens of comments on Hacker News this week. Here is what it does, what the evidence supports, and why the idea matters more than this one project.

The trick

The models involved are mixture-of-experts models. They are built from many small specialist blocks, called experts, and for each word generated only a few of them are used. The file is enormous, but any single step touches a fraction of it. So the engine keeps the always-needed parts in memory, reads the experts from a fast SSD when they are called, and keeps the ones that keep coming back in a cache.

Ordinary "dense" models can't do this, because every weight is used for every word. The model Janas was built around, Qwen3-Next-80B-A3B, is a 48 GB file. The "A3B" in the name refers to roughly three billion parameters being active at a time.

What it reports

On the author's laptop (an Intel Core Ultra 9 with 32 GB of RAM and an NVMe SSD reading at about 5.7 GB/s), the 80B model answers at roughly 20 to 25 tokens per second, and a smaller 35B model at about 32.
Using the model's own built-in drafting mechanism, the headline figure rises to 37 tokens per second.
In the author's comparison, llama.cpp, the best-known local engine, manages 2.8 tokens per second on the same model, and reads prompts at 16 against Janas's 62. He says llama.cpp runs out of memory once the GPU is involved.
With a browser and an editor open and more than 8 GB already in use, speed dropped to nearly 18.

What it doesn't show

One machine. The author says everything was developed and measured on a single laptop. AMD processors, chips without a particular instruction set, machines with 16 GB or less, and slower SATA disks are all untested. Windows and macOS aren't supported.
Mostly one model. Nearly all the measurements are on the 80B model. The others were converted and checked, but far less.
Stability is unverified. The engine fills free memory with its cache and reads the disk at full speed. On a machine with less headroom than it expects, the desktop can slow to a crawl. Once, using the GPU caused a driver reset that took the desktop down.
It's an experiment. The author says so himself, and says commands and file formats have changed several times a week. When we looked, the public repository had a single commit.
The comparison is the author's. The llama.cpp numbers use his configuration. By one automated summary, about half of the commenters were sceptical, questioning the benchmark choices and whether this duplicates features llama.cpp already has, and about a fifth were positive.

It isn't just one project

The technique is showing up in several places at once. A pull request to llama.cpp proposes streaming mixture-of-experts weights from disk in the same way. A fork called Slipstream does it for Macs, aiming to run coding models larger than the machine's memory. Other Apple-silicon projects take similar approaches. Whether or not Janas itself lasts, the idea seems to be spreading.

Our read

The useful point is that, for this kind of model, a fast SSD can stand in for part of your RAM, which changes what a given machine can run. The rule of thumb in our post on how much memory an AI model needs still applies to dense models, but mixture-of-experts models bend it.

The limits are equally clear. It only works for mixture-of-experts models, it depends on a quick disk, and it is early software. We haven't seen it tried on a phone, and phones have far less memory than laptops, so we wouldn't assume it carries over. That is our speculation, not something the project claims.

If you want to try it

Janas's own README advises starting on an otherwise idle machine and capping the memory it takes with its cache option, since it is designed to use whatever is free. Treat it as something to experiment with, not something to put on your daily computer.

What to watch

Whether people with other hardware reproduce the speeds, especially on machines with less memory or slower disks.
Whether llama.cpp merges its own version of SSD streaming.
Whether the approach reaches Macs and other mainstream setups reliably.

For the hardware side of running models at home, see Local AI Hardware in Autumn 2026.

Sources

For disclosure: we make iPhone apps, OffgridStem, OffgridScribe, OffgridVox and OffgridCam. We haven't run Janas ourselves.

More from Offgrid Studio

Can a Linux Phone Replace Your iPhone? A 2026 Check
What a Hospital for Coding Agents Teaches About Review

Back to the blog