Bonsai 27B shrank from 54 GB to 3.9 GB while keeping math and coding surprisingly strong — but tool use and instruction following took a much larger hit.
Press enter or click to view image in full size
Not a Medium member? You can read the full article for free using this friend link
A phone running an AI model is no longer unusual. Apple already ships small models on the iPhone, and open models such as Gemma can run on mobile hardware.
These models are useful for autocomplete, summarization, classification, and simple question answering. Some can also call tools or follow short workflows.
The challenge starts when the task becomes longer.
A model may need to plan several steps, call multiple tools, return valid structured output, recover from an error, and remember its original goal. Smaller models can do this, but their reliability often drops as the workflow grows.
That is why PrismML’s Bonsai 27B is worth paying attention to.
The company says it compressed a 27-billion-parameter model enough to run directly on an iPhone 17 Pro Max, with inference happening on the phone rather than in the cloud.