SerpApi.Org
Artificial intelligence models have grown larger every year, but so have their hardware demands. Running advanced AI has typically required powerful servers, desktop GPUs, or expensive workstations equipped with large amounts of memory. That has made cutting-edge models inaccessible to many users who simply want AI to work on their everyday devices.
PrismML is now challenging that assumption with the release of Bonsai 27B, an AI model that packs 27 billion parameters into a package small enough to fit on a modern smartphone. Instead of relying on cloud computing, the model runs locally, marking an important step toward bringing more capable AI directly to consumer hardware.
Why do large AI models normally need so much memory
Parameters are the internal values that allow an AI model to recognize patterns, generate responses, write code, answer questions, and perform reasoning tasks. In general, more parameters give a model greater capacity to understand complex prompts and produce higher-quality results.
The downside is that every parameter requires memory. A conventional 27-billion-parameter model stored in half-precision requires roughly 54 GB of memory before it can even begin processing requests. That exceeds the memory available in virtually every smartphone and even many consumer laptops.
As AI models continue to grow, hardware requirements have become a major barrier to wider adoption. Developers have spent years searching for ways to reduce those requirements without sacrificing performance.
Bonsai 27B takes a different approach
PrismML's breakthrough comes from dramatically changing how model weights are stored. Traditional full-precision AI models often represent each weight using 16 bits of precision. Bonsai replaces that system with an extremely compressed format. The binary version reduces each weight to either positive or negative, while the ternary version adds a third value, zero.
Rather than storing thousands of possible numerical values for every weight, the compressed model uses low-bit weights with group-wise scaling to preserve much of the model's behavior. The binary version uses 1.125 effective bits per weight, while the ternary version uses 1.71 effective bits per weight.
The result is remarkable. The binary Bonsai 27B occupies only 3.9 GB, while the ternary version requires about 5.9 GB. That makes the binary model roughly 14 times smaller than its original 16-bit version.
Compression without the usual compromises
Many compressed AI models still keep certain sensitive components at higher precision to avoid hurting performance. Those exceptions increase storage requirements and reduce the overall benefit of compression. Bonsai takes a more aggressive approach.
According to PrismML, embeddings, attention layers, and the language model head are compressed together rather than allowing selected components to remain larger. Compressing the model end-to-end keeps memory requirements low while maintaining competitive performance across a wide range of tasks.
The technology is based on intellectual property developed at Caltech and represents a different philosophy from conventional low-bit AI optimization techniques.
Running directly on an iPhone
One of the most eye-catching achievements is that Bonsai 27B can operate on a high-end iPhone. The 3.9 GB binary version reportedly runs at around 11 tokens per second on an iPhone 17 Pro.
Tokens are the pieces of information an AI processes when reading or generating text. PrismML also reports much faster generation on laptop-class Apple Silicon, including up to 58 tokens per second for the ternary model on an M5 Max.
Instead of sending prompts to cloud servers, users can execute the model directly on their own hardware. That reduces dependence on internet connectivity while giving users greater control over where their information is processed. For developers building privacy-focused applications, local execution could become an increasingly attractive option.
Performance stays surprisingly close to full-size models
Shrinking an AI model is only useful if it still performs well. Across 15 benchmarks covering knowledge, reasoning, math, coding, instruction-following, tool-calling, and vision, the ternary version of Bonsai 27B averaged 80.5, retaining 95% of the full-precision baseline. The 1-bit version posted an average score of 76.1 and retained 90% of the full-precision baseline.
The benchmark results also suggest the model delivers impressive efficiency relative to its size. PrismML reported strong results across coding, mathematics, and general knowledge while using only a fraction of the memory typically required by models in this class.
Rather than competing solely on raw benchmark numbers, Bonsai demonstrates how much capability can now fit into extremely limited hardware resources.
A new architecture helps expand context
Another important feature is Bonsai's long-context design. PrismML says Bonsai 27B carries a full 262K-token context window, helping the model handle longer documents, larger conversations, and extended agentic workflows.
Large context windows enable AI models to analyze longer documents, retain more information during conversations, and handle larger projects without losing track of earlier details. Achieving that capability on consumer hardware would normally require substantially greater computing resources.
Testing shows practical strengths
PrismML says Bonsai 27B was evaluated across practical capability areas, including coding, tool use, reasoning, math, instruction-following, knowledge, and vision.
The company reported strong benchmark results in coding and math, with the ternary model scoring 86.0 in coding and 93.4 in math. PrismML also said the model supports structured tool calls, vision tasks, and multi-step agentic loops that stay coherent across many steps.
These results indicate that while cloud-based frontier models still hold an advantage in many areas, local AI is steadily becoming capable enough for everyday development and productivity tasks.
Faster generation through speculative decoding
PrismML also includes DSpark, a speculative decoding system designed to increase generation speed. Instead of generating text, only one token at a time, a lightweight drafting model predicts possible tokens, while the primary model verifies those predictions.
According to PrismML’s Hugging Face documentation, DSpark uses confidence-scheduled verification and preserves the target distribution exactly, so accepted tokens are indistinguishable from those generated by ordinary methods. PrismML’s Bonsai 27B page says the model supports speculative decoding, compounding speed with lossless draft-and-verify acceleration.
Apple's reported interest adds another dimension
The announcement also drew attention because PrismML CEO Babak Hassibi told CNBC that the company is in early discussions with Apple regarding its compression technology.
While no agreement has been announced, Apple's long-term emphasis on on-device intelligence makes the technology a potentially interesting fit. The company has increasingly focused on processing AI workloads locally whenever possible to improve privacy, reduce latency, and lessen dependence on cloud infrastructure.
If compression techniques like Bonsai become more widely adopted, future smartphones may be able to run increasingly capable AI systems without requiring remote servers for every request.
The bigger picture for open AI development
Bonsai 27B arrives amid renewed interest in open-weight AI models. Thinking Machines, the startup founded by former OpenAI Chief Technology Officer Mira Murati, also introduced a new open-weight model called Inkling. With 975 billion parameters, Inkling targets developers who want to download, run, and customize AI systems rather than relying entirely on proprietary services.
Although Inkling operates at a vastly different scale than Bonsai, both announcements highlight an important trend. Companies are exploring new ways to make advanced AI more accessible, whether through massive open-weight models for developers or highly compressed systems that run directly on consumer devices.
Rather than focusing only on building larger models, the industry is increasingly investing in making powerful AI practical across a broader range of hardware.
What this breakthrough could mean next
Bonsai 27B demonstrates that AI innovation is no longer defined solely by adding more parameters. Efficient model design is becoming just as important as raw size.
If future compression techniques continue to improve while preserving quality, smartphones, tablets, and lightweight laptops could handle increasingly sophisticated AI workloads without relying on constant internet access. That shift would improve privacy, lower operating costs, and expand access to advanced AI capabilities for millions of users.
The release also suggests that the next stage of AI competition may revolve around efficiency as much as intelligence. The companies that learn to deliver powerful performance within tight hardware limits could shape how AI reaches everyday consumers in the years ahead.
Why this matters for everyday users
Most people will never own a server packed with expensive GPUs, but nearly everyone carries a smartphone. Bonsai 27B shows that advanced AI is steadily moving from specialized hardware into devices people already use every day.
As model compression improves and mobile processors become more capable, running sophisticated AI locally could become a standard feature instead of a premium capability. That would represent one of the biggest shifts in consumer AI since the rise of cloud-based chatbots.
TL;DR
- PrismML released Bonsai 27B, the first 27B-class AI model designed to run on a phone.
- The binary version occupies just 3.9 GB, compared with roughly 54 GB for a conventional 16-bit model of similar size.
- The compression method reduces model weights to binary or ternary values while preserving most performance.
- Bonsai reportedly runs at about 11 tokens per second on an iPhone 17 Pro.
- The ternary version of the model retains about 95% of the full-precision model's benchmark performance.
- Bonsai 27B supports a 262K-token context window.
- PrismML says it is in early discussions with Apple regarding its compression technology.
- The release highlights the growing importance of efficient on-device AI alongside larger cloud-based models.
Don’t forget to follow us for more exclusive content.
If you liked this, you might also like:
- Android features Apple should bring to iPhone 18
- Apple Watch Ultra 3 May Be Driving Users Away
- Meta’s smart glasses draw privacy concerns after code discovery
This article was made with AI assistance and human editing.
