PrismML Fit a 27B Model Into 3.9GB and Put It on a Phone

The on-device AI conversation has been stuck on a bad trade for two years. You could have a small model that fits on a phone and is dumb, or a capable model that lives in a data center and bills you per token. Every "AI on your device" launch since has been a negotiation with that constraint — a 3B here, a distilled 8B there, each one an apology for what it couldn't fit.

On July 14, PrismML released Bonsai 27B and stopped apologizing. It is a full 27-billion-parameter model, compressed to 3.9GB, running on an iPhone 17 Pro Max at about 11 tokens per second. Not a distillation. Not a smaller sibling. The 27B, quantized down to 1.125 bits per weight, with the whole thing under Apache 2.0.

What they actually did

Bonsai 27B is built on Qwen3.6-27B — Alibaba's open-weight base, currently the most-derived-from model family outside the American labs. PrismML ships two variants:

• Ternary, at 1.71 bits per weight, weighing 5.9GB, retaining 94.6% of the FP16 baseline.
• 1-bit binary, at 1.125 bits per weight, weighing 3.9GB, retaining 89.5%.

The quantization is unusually total. Weights are reduced to ternary {−1, 0, +1} or binary {−1, +1} values, with a single shared FP16 scale per group of 128 — and critically, the low-bit representation runs end to end. Embeddings, attention, the MLPs, and the language model head all use the compressed format. Most aggressive quantization schemes leave the sensitive layers in higher precision and quietly reclaim a chunk of the savings; PrismML didn't.

Both variants are multimodal. The vision tower is a compact 0.46B parameters at 4-bit, which means on-device workflows can read screenshots, documents, and camera input rather than text alone. Context runs to 262K tokens. On an M5 Max, the 1-bit build hits 66.4 tokens/sec generating and 874 on prefill; an M5 Pro running ternary does 26.2 and 393.

Those laptop numbers are comfortable. The phone number is the one that matters: 11 tokens a second is slower than reading speed but well inside the range where a local assistant is usable rather than a demo. And it is happening in 3.9GB, on hardware people already own.

The 89.5% question

The honest reading requires sitting with that retention figure. The 1-bit build gives up more than ten percent of the base model's benchmark performance. That is not a rounding error, and anyone selling this as free compression is selling something.

But "10% worse than Qwen3.6-27B" is the wrong comparison. The right one is against what else fits in 3.9GB — which, until this week, was a 3B or 4B model. Against that baseline, a 27B at 89.5% is not a degradation; it's a category change. The ternary build makes the trade even more interesting: 94.6% retention for 5.9GB, which is still laptop-trivial and phone-plausible on high-memory devices. You are choosing where on the curve to sit, and both points on that curve were unreachable a week ago.

The speculative-decoding support compounds this. Draft-and-verify acceleration is lossless — it doesn't change what the model outputs, only how fast it gets there — so the throughput numbers above are a floor, not a ceiling.

Why this lands now

The timing is not a coincidence, and it connects to a story most coverage ran separately. The same week PrismML published Bonsai, China's Cyberspace Administration approved Apple Intelligence for the mainland with Alibaba's Qwen underneath it. PrismML's compression is what makes a 27B-class Qwen viable in phone memory — roughly 54GB reduced to under 4GB, running on hardware as old as an iPhone 15 — and Apple is reported to be in talks with the company about the technology.

PrismML itself is a Khosla Ventures-backed spinout from Caltech. Small team, one released artifact, and a permissive license on a result that large device makers have been trying to buy their way to.

That Apple would be interested is not surprising. Apple's entire strategic position — the privacy envelope, the silicon integration, the reluctance to ship user data to someone else's inference cluster — depends on capable local inference existing. Apple has spent two years conceding the model layer to partners precisely because the models it could run locally weren't good enough to concede nothing. Compression like this is the thing that would let Apple stop conceding.

What it breaks

The commercial logic of a lot of AI products assumes inference is a metered utility. If a 27B-class multimodal model with a 262K context runs on a phone for free, forever, offline, then the floor under per-token pricing for an enormous class of workloads — summarization, extraction, classification, document QA, the unglamorous majority of what people actually ship — is not a lower price. It's zero.

Apache 2.0 is what makes that real rather than rhetorical. There is no license negotiation, no per-device fee, no vendor to route around. Anyone building an app can ship this inside the binary.

The catch is that 89.5% is not 100%, and the workloads where the last ten percent matters — hard reasoning, long-horizon agentic work, anything where a subtle error compounds — are exactly the workloads the frontier labs price highest. The cloud isn't losing those. What it's losing is the long tail underneath them, the volume business that was always a bad fit for a metered API and stayed there only because nothing else fit in memory.

PrismML didn't beat the frontier. It made the frontier's leftovers free, and put them in everyone's pocket under a license nobody has to ask about.