PrismML hopes its little LLM will change the way we all use AI

If the AI ​​laboratory PrismaML isn’t on your radar yet, it should be, not because it’s raised huge amounts of money (it hasn’t yet, just a $22.25 million seed round), but because of the technical minds involved and the technology it’s developing that will potentially change the industry.

PrismML is betting that large, capable, high-performance, reasoning language models don’t have to be large.

It is making reasoning models so small they fit on PCs and smartphones. (It is even rumored be in talks with Apple(although CEO Babak Hassibi declined to comment to TechCrunch).

On Thursday, PrismML released Bonsai 2 27Bthe latest in a family of models, compressing Qwen3.8 27B, a widely used open source model from Alibaba, up to 5.9GB. It’s small enough to fit on a PC and possibly a high-end smartphone. It’s a 9-10x memory reduction compared to the original.

PrismML was founded by a group of Caltech researchers and is led by Hassibi, a Caltech professor and expert in compression technologies. The startup also has Ion Stoica as an advisor. Stoica is co-founder of Databricks (and other companies) and director of the famous Sky Computing Lab in Berkeley, from which many technologies and startups have emerged, from letta to SGLang.

PrismML is also backed by investors Khosla Ventures, Cerberus Capital and Caltech.

This startup is certainly not the only company working on LLM compression technology. Multiverse Computing, founded by a well-known professor at the International Physics Center of Donostia (Spain), is another. (And multiverse computing has raised tons of cash.)

But Hassibi says PrismML’s compression technology is unique because its LLMs have lost virtually no performance compared to the originals. Bonsai 2 matches 98% of Qwen’s aggregated benchmark scores. That’s more than the first Bonsai, released a couple of months ago in March, which equaled 95%. That original model has already been downloaded more than 11 million times, and PrismML’s even smaller models have been downloaded another 2.6 million times, the company says.

This shows that PrismML’s compression results have improved from one version to the next. It remains to be seen whether 100% benchmark performance parity can one day be achieved. Compression is likely to always have some impact, says Hassibi.

Still, perfect reference parity is pretty academic anyway. LLMs are not as accurate in their uncompressed form, and benchmarks do not reflect actual tasks so perfectly, that a 2% degradation would likely significantly impact a model’s performance in real-world use. (Also, the surrounding software (the harness on which a model runs) matters a lot when it comes to precisionalso.)

PrismML says it achieves this by reducing the “weights” that make up a model; Weights are essentially the information that a model learns and stores during training. Typically, each weight requires 16 bits. PrismML’s approach, called “ternary” weights, simplifies this to three: +1, −1, or 0. With much smaller values ​​to store for each weight, the model takes up much less space. (For a deeper dive into the compression technique, here is the project GitHub page.)

The startup’s next goal is to apply this compression technique to even larger models. “The next models that we will release, hopefully in the next few months, will be in the range of several hundred billion parameters, and I hope it will be easier to retain the intelligence there,” Hassibi told TechCrunch.

As model size grows, he added, “there’s more room to be able to compress them without losing intelligence. So I would just say, as a general trend, that for larger models, it’s easier to get to 100%.”

Stoica tells us that he is excited about this technology because it makes it possible for advanced models to run on users’ devices. “You’ll have intelligence at your fingertips, and it will be free because it will run on the device you already purchased. It will also be private, because you won’t send it to the cloud.”