A 35-billion-parameter model now runs on a $599 Mac Mini. That doesn't make it useful for everything.
Ternary quantization gets a 35B open-weight model running on a base Mac Mini at reading speed. It still confidently got GEO wrong, which is the more useful finding.
Ben Rees - 30 July 2026
A stock 16GB Mac Mini, the cheapest one Apple sells, can now run a 35-billion-parameter language model entirely in memory. No internet connection, nothing streamed from disk. I got mine running the week before I left for holiday, and the moment I got back and asked it something I actually cared about, it confidently told me GEO stands for "Global Economic Opinion."
Both of those things are true. The first is a genuinely impressive piece of engineering. The second is why I don't think local models replace anything I currently pay for. They do something different.
The trick is compression, not a smaller model
The model is Qwen3.6-35B, an open-weight mixture-of-experts model from Alibaba's Qwen team. Uncompressed it's about 70GB, nowhere close to fitting in 16GB of memory. Manjunath Janardhan worked out how to get it down to 9.4GB using a tool he built called TurboQuant-MLX, and I followed his recipe on my own machine rather than trusting the headline number.
The compression is aggressive. Every weight in the model's 256 "expert" sub-networks gets stored as one of three values: -c, 0, or +c. Three possible values is log₂3, roughly 1.58 bits per weight, the same ternary scheme Microsoft Research described in their BitNet b1.58 paper on training language models this way from scratch. Janardhan applied it after the fact, to weights that were never trained expecting it, and it still works because the parts that can't tolerate rough treatment, the attention layers and the routing logic, stay at full precision. Only the experts get squeezed.
On my own Mac Mini: about 10 tokens a second, roughly reading speed, using 10.4GB of memory. That matched what Janardhan measured almost exactly.
Then I asked it something I actually cared about
Once it was running, I didn't ask it to explain quantum computing. I asked what GEO means in B2B marketing, a term I've built an entire measurement system around tracking for my own content.
It told me GEO stands for "Global Economic Opinion." Confidently. No hedging, no uncertainty flagged. Just wrong.
That's not a bug in the compression. A frontier cloud model would probably get GEO right, because it's been trained on enough recent, specific content about generative engine optimisation to have formed a real association. A 35B open-weight model, squeezed to 1.58 bits per weight and running entirely offline with no access to anything published after its training cutoff, was never going to know a term that's still relatively niche. Compression didn't cause the gap. It just made the gap cheap enough to hit on a $599 machine instead of a five-figure GPU workstation.
What this is actually for
I didn't buy a Mac Mini to replace Claude or ChatGPT. The same logic I've written about for why GEO and AEO are different problems applies here: a local model is a good tool for the narrow set of tasks where privacy, cost, or offline access matters more than being right about anything specific. Drafting. Summarising. Formatting. The parts of work that don't depend on current or specific knowledge.
Everything that does depend on that, and most of what I actually do day to day does, still needs to go to a model with real training data currency, or better yet, retrieval against something you know is true. That's not a limitation of this particular model. It's a structural fact about what gets stored inside a model's weights versus what has to be looked up.
The interesting problem now isn't whether a local model can run. It clearly can. A year ago, when I first tried this with Ollama on an ordinary laptop, it was a much smaller model at a fraction of the speed. The interesting problem is building something that knows, automatically, which questions are safe to hand to the cheap machine on the desk and which ones need to go to the expensive one in the cloud.
That's what I'm building next.
Related reading
My Mac Mini is going back to the shop
Three separate experiments on the same Mac Mini all pointed at the same conclusion: free and private don't matter if the answers are wrong or don't arrive at all.
I ran a local AI model through 179 rounds of self-refine. It collapsed. One sampler setting fixed it.
An unguarded local model collapsed into repetition after 179 rounds of self-refine. Adding one sampler setting and rerunning the identical task produced 201 clean cycles instead.
Self-Refine Without a Repetition Penalty
A controlled case study: the same recursive self-improvement loop, on the same local model, six and a half hours each. One run collapses into repetition within forty cycles. The other holds clean for all two hundred and one. The difference is a single sampler setting.
Human Beings are Holding Back Machine Learning
Machine learning tools have been accessible for decades. The real shortage has always been people who know how to use them properly.