How to Run Bonsai 27B on Your Phone (iPhone & Android, Step by Step)
Quick answers:
- A 12GB+ phone runs the 1-bit build comfortably at a normal 4K context. It also fits many 8GB phones, but with little headroom.
- Use the 1-bit Q1_0 build for this guide. The ternary file can fit a 16GB Android in hardware, but current PocketPal releases do not load that tracked format.
- Total install time is ~15 minutes, most of it the 3.8GB download — use Wi-Fi.
- Expect roughly 8–10 tokens/s (estimated) on 2025–26 flagships — faster than you read.
- Not sure your phone qualifies? Check it first — that's step 1.
Step 1 — Confirm your phone can run it
Bonsai 27B's 1-bit build needs about 4.8GB of usable memory at 4K context (3.8GB file + weight, KV-cache, and runtime overhead) — and usable memory is much less than the RAM on the spec sheet. iOS caps a single app at ~65% of total RAM; Android reserves 2–4GB for the system.
The quick rule:
| Your phone | 1-bit (3.8GB) | Ternary (7.2GB) |
|---|---|---|
| 16GB Android (Galaxy S26 Ultra, OnePlus 15…) | ✅ | ✅ |
| 12GB Android / iPhone 17 Pro & Pro Max | ✅ | ❌ |
| 8GB phones (iPhone 16 Pro, Galaxy S24…) | ⚠️ fits at 4K with limited headroom | ❌ |
The table above is a memory-fit check. It does not mean the current phone app supports both file formats: the step-by-step path below is for Q1_0 only.
Full per-phone verdicts: Bonsai 27B on the phones we track, or jump straight to your phone, e.g. Galaxy S26 Ultra or iPhone 17 Pro Max.
Step 2 — Pick your build
- 1-bit (Q1_0, 3.8GB) — the one most people should download. Fits 12GB phones with headroom.
- Ternary (Q2_0, 7.2GB) — the memory can fit a 16GB Android, but PocketPal does not support this tracked file today. Do not download it for this walkthrough.
- On a smaller phone, use a conventional model with a current phone-app path, such as Qwen3.5 4B or Llama 3.2 3B.
Step 3 — Install a local AI app
- iPhone & Android: PocketPal AI — free, open-source, no account. Install or update it from the App Store / Play Store.
Step 4 — Download the model
In PocketPal's model hub, search prism-ml/Bonsai-27B-gguf and choose
Bonsai-27B-Q1_0.gguf. On Wi-Fi the 3.8GB download typically takes a few minutes.
Do not choose F16, mtp-*, mmproj-*, or other helper files for this text-chat walkthrough.
Step 5 — Set context to 4K (and Metal on iPhone)
Before the first chat:
- Context length: 4096. These memory numbers assume 4K; Bonsai's hybrid-attention layout adds about 0.25GB of KV cache for every extra 4K of context.
- iPhone: keep PocketPal's default Metal acceleration. Metal is the inference backend inside the app; it does not require Xcode.
- Close your other apps — on a 12GB iPhone the estimate leaves about 3GB of working headroom after loading, but iOS can still evict the model if other apps consume that budget.
Step 6 — First run: what to expect
Our bandwidth-based estimates (formula, not measured — methodology):
| Phone | Est. speed |
|---|---|
| Galaxy S26 Ultra / OnePlus 15 | ~10 tokens/s |
| iPhone 17 Pro Max / Galaxy S24 Ultra | ~9 tokens/s |
| Pixel 10 Pro / POCO X7 Pro | ~8 tokens/s |
PrismML's own figure for iPhone 17 Pro Max is ~11 tokens/s. Anything above ~8 tokens/s reads faster than you can read; a ~300-word reply lands in under a minute. The first response after loading is slower — that's prompt processing, not generation.
If something goes wrong
- App crashes while loading → not enough free memory. Reboot, load again before opening anything else, or drop context to 2K. If it still closes, use a smaller conventional model such as Qwen3.5 4B.
- Starts fast, gets slower → thermal throttling; sustained generation heats the chassis within ~10 minutes. Take the case off, pause between long replies.
- Gibberish or repetition → re-download the file (corrupted download) and make sure the app is up to date — Bonsai's quant formats need a recent llama.cpp build.
FAQ
Does it work fully offline? Yes — after the download, inference is on-device. Airplane mode works.
Can it see images? The model family is multimodal, but this guide covers text chat. Image input needs a compatible vision-projector path; the main Q1_0 GGUF alone does not make every phone app image-ready.
Can I use the ternary file in PocketPal? Not with the current tracked phone path. Its memory fit is real, but its runtime format is separate. Use Q1_0 for the PocketPal walkthrough.
Is 1-bit quality actually usable? PrismML reports ~90% of full-precision performance across 15 benchmarks. Independent verification is still thin — for coding, compare it against Qwen3.6 27B's smaller siblings on your own prompts. See our Bonsai vs Qwen3.6 comparison.
What about the requirements in detail? Full RAM math per variant and per phone: Can your phone run Bonsai 27B?
Speeds are formula estimates from our fit engine unless marked measured; measured entries land as the community submits them.