A 35-billion-parameter model now runs on a $599 Mac Mini. That doesn't make it useful for everything.
Ternary quantization gets a 35B open-weight model running on a base Mac Mini at reading speed. It still confidently got GEO wrong, which is the more useful finding.
Ben Rees - 30 July 2026
A stock 16GB Mac Mini, the cheapest one Apple sells, can now run a 35-billion-parameter language model entirely in memory. No internet connection, nothing streamed from disk. I got mine running the week before I left for holiday, and the moment I got back and asked it something I actually cared about, it confidently told me GEO stands for "Global Economic Opinion."
Both of those things are true. The first is a genuinely impressive piece of engineering. The second is why I don't think local models replace anything I currently pay for. They do something different.
The trick is compression, not a smaller model
The model is Qwen3.6-35B, an open-weight mixture-of-experts model from Alibaba's Qwen team. Uncompressed it's about 70GB, nowhere close to fitting in 16GB of memory. Manjunath Janardhan worked out how to get it down to 9.4GB using a tool he built called TurboQuant-MLX, and I followed his recipe on my own machine rather than trusting the headline number.
The compression is aggressive. Every weight in the model's 256 "expert" sub-networks gets stored as one of three values: -c, 0, or +c. Three possible values is log₂3, roughly 1.58 bits per weight, the same ternary scheme Microsoft Research described in their BitNet b1.58 paper on training language models this way from scratch. Janardhan applied it after the fact, to weights that were never trained expecting it, and it still works because the parts that can't tolerate rough treatment, the attention layers and the routing logic, stay at full precision. Only the experts get squeezed.
On my own Mac Mini: about 10 tokens a second, roughly reading speed, using 10.4GB of memory. That matched what Janardhan measured almost exactly.
Then I asked it something I actually cared about
Once it was running, I didn't ask it to explain quantum computing. I asked what GEO means in B2B marketing, a term I've built an entire measurement system around tracking for my own content.
It told me GEO stands for "Global Economic Opinion." Confidently. No hedging, no uncertainty flagged. Just wrong.
That's not a bug in the compression. A frontier cloud model would probably get GEO right, because it's been trained on enough recent, specific content about generative engine optimisation to have formed a real association. A 35B open-weight model, squeezed to 1.58 bits per weight and running entirely offline with no access to anything published after its training cutoff, was never going to know a term that's still relatively niche. Compression didn't cause the gap. It just made the gap cheap enough to hit on a $599 machine instead of a five-figure GPU workstation.
What this is actually for
I didn't buy a Mac Mini to replace Claude or ChatGPT. The same logic I've written about for why GEO and AEO are different problems applies here: a local model is a good tool for the narrow set of tasks where privacy, cost, or offline access matters more than being right about anything specific. Drafting. Summarising. Formatting. The parts of work that don't depend on current or specific knowledge.
Everything that does depend on that, and most of what I actually do day to day does, still needs to go to a model with real training data currency, or better yet, retrieval against something you know is true. That's not a limitation of this particular model. It's a structural fact about what gets stored inside a model's weights versus what has to be looked up.
The interesting problem now isn't whether a local model can run. It clearly can. A year ago, when I first tried this with Ollama on an ordinary laptop, it was a much smaller model at a fraction of the speed. The interesting problem is building something that knows, automatically, which questions are safe to hand to the cheap machine on the desk and which ones need to go to the expensive one in the cloud.
That's what I'm building next.
Related reading
Human Beings are Holding Back Machine Learning
Machine learning tools have been accessible for decades. The real shortage has always been people who know how to use them properly.
Why AI systems cite some SMB-to-enterprise stories and ignore others
AI systems rarely cite customer case studies that only live on the vendor's own site. Citation comes from independent corroboration, not content quality - here's what to do about it.
A Weighted Sum of Everything Ever Written About You
What Bayesian linear regression, copied by hand from a textbook I've owned since January 2007, tells us about getting cited by AI.
Your brand doesn't get its own space inside an AI model
Language models pack far more concepts than they have space for, a phenomenon called superposition. Generic content competes for that same overcrowded space as everyone else's, and usually loses.