Bonsai 27B vs Qwen3.6 27B on Phones: Is 1-Bit Worth It?
Quick answers:
- On a phone, Bonsai wins by default — only 24GB Androids (OnePlus 13, ROG Phone 9 Pro) can load Qwen3.6 27B. Q3_K_M needs ~16.8GB and runs at ~2.5 tokens/s; Q4_K_S also fits at ~19.2GB and ~2.2 tokens/s (est.).
- Bonsai 27B 1-bit (3.8GB) runs on any 12GB+ phone at ~8–10 tokens/s (est.) — same model class, 4–5× smaller footprint, roughly 4× faster decode.
- The real trade is quality: PrismML reports the 1-bit build keeps ~90% of full-precision performance. If your workload needs the last 10% — heavy coding, long precise reasoning — run Qwen3.6 in the cloud or on a PC, not on a phone.
- Ternary Bonsai (7.2GB, 16GB Androids) sits between the two: more bits than 1-bit, still 6GB+ smaller than any Qwen3.6 quant.
Why these two are directly comparable
Bonsai 27B isn't a random competitor — PrismML distilled it from Qwen3.6-27B, quantizing end-to-end to 1.125 bits/weight (1-bit) or 1.71 (ternary). So this is the closest thing to a controlled experiment: the same 27B-class knowledge, packaged for phone memory budgets vs. packaged conventionally.
The numbers side by side
| Bonsai 27B 1-bit | Bonsai 27B ternary | Qwen3.6 27B Q3_K_M | Qwen3.6 27B Q4_K_S | |
|---|---|---|---|---|
| Download | 3.8 GB | 7.2 GB | 13.6 GB | 15.9 GB |
| Usable RAM needed* | ~6.5 GB | ~10.1 GB | ~16.8 GB | ~19.2 GB |
| Phones that fit | 12GB+ | 16GB Androids | 24GB Androids only | 24GB Androids only |
| Est. speed on S26 Ultra | ~10 tokens/s | ~5.3 tokens/s | won't fit | won't fit |
| Est. speed on OnePlus 13 (24GB) | ~9 tokens/s | ~4.8 tokens/s | ~2.5 tokens/s | ~2.2 tokens/s |
*At 4K context: file × 1.05 + KV cache + runtime, per our methodology. Speeds are bandwidth-based estimates, not measurements.
Decode speed on phones is memory-bandwidth-bound — every token reads the whole model once. A 13.6GB file simply cannot decode fast on a phone bus: that's why Qwen3.6's ~2.5 tokens/s isn't a tuning problem, it's physics. What 2.5 tokens/s feels like →
What you give up with 1-bit
PrismML's own benchmarks put the 1-bit build at ~90% of full precision across 15 benchmarks — but extreme quantization quality is workload-dependent, and independent phone-side evaluations are still scarce. Practical guidance:
- Chat, summarization, translation, general Q&A: the 90% claim is plausible; most users won't feel the gap.
- Coding and strict formatting: test on your own prompts before trusting it. A dense Qwen3.5 4B or Ministral 3 8B at Q4 can beat a heavily-quantized 27B on precision tasks while being faster.
- Long precise reasoning: if it's worth real money, use the full model off-device.
Verdict by device
- 12GB phone (iPhone 17 Pro, Galaxy S24 Ultra, Pixel 9…): Bonsai 1-bit is the only 27B-class option, and it's genuinely usable — check your exact phone.
- 16GB Android (S26 Ultra, OnePlus 15…): Bonsai 1-bit for speed, ternary if you'll trade half the speed for extra quality headroom.
- 24GB Android (OnePlus 13, ROG Phone 9 Pro): you can load Qwen3.6 27B, up to Q4_K_S at ~2.2 tokens/s (or Q3_K_M at ~2.5). It is still more party trick than daily driver.
- PC or cloud: run Qwen3.6 27B at Q4+ and skip the compromise entirely.
FAQ
Is Bonsai 27B actually the same model as Qwen3.6 27B? It's distilled from it — same class and lineage, but a separate set of weights trained to survive 1-bit quantization. It is not "Qwen3.6 with a smaller file".
Why not just run Qwen3.6 27B at Q2? A conventional Q2_K of a 27B is still ~10GB and conventional 2-bit quantization degrades quality sharply. Bonsai was trained for low-bit inference — that's the whole point of quantization-aware distillation.
What about Qwen3.6 35B-A3B (MoE)? Its 3B active parameters would decode fast, but the whole model still has to sit in memory: the smallest quant is ~16.6GB, which misses the cut even on 24GB Androids once KV cache and runtime are counted. On phones it's not an option; on a PC it's excellent.
Which phones run Bonsai, exactly? Per-variant, per-phone tables: Can your phone run Bonsai 27B? — or the live pages for 1-bit and ternary.
Estimates from our fit engine (bandwidth ÷ bytes-per-token with a conservative efficiency factor); quality figures are PrismML's published numbers, labeled as such. Measured entries will replace estimates as community benchmarks land.