The days of Nvidia’s unparalleled market dominance are not over, but challenges and options are emerging from all directions.
ZMLa French fashion AI startup endorsed by Turing Award winner Yann LeCun, has launched inference-performance software that allows a variety of large open source language models run in a variety of fries – including Nvidia, AMD, Google TPU, Apple Metal and Intel Arc.
With ZML/LLMDWith the recently launched LLM inference server, the company’s ambition is to break down existing silos and make different chips available for AI use cases at their maximum available speed and sometimes faster, ZML founder Steeve Morin told TechCrunch.
As AI becomes integrated into our work and daily lives, optimizing inference (also known as prompt processing) has been outpacing model training in importance, but it often feels patchy behind the scenes, with software and architectural barriers that lead to vendor lock-in, Morin said.
The promise of achieving maximum performance in a variety of chips is a technological feat, but it could also be a market disruptor, amid growing fears about AI-related costs.
ZML hopes to offer enterprises and clouds the option of using a combination of chips, some of which could be less expensive or consume less power. “The idea is to give people the power to create their own system and achieve real efficiency gains that allow [AI] to be disseminated,” Morin said.
Such software support could help new AI chip makers, many of which are from Europe, Morin noted, citing Axelera, fractile, Kalray, OLIX, Q.ANT, siperla, Spinncloudand VSORA. But more than their region of origin, what matters to them is that ZML can work with them on “things that have not been done before anywhere in the world.”
That doesn’t mean Morin is bearish on Nvidia. he is notpartly due to its existing offering. He told TechCrunch that ZML has a good relationship with the AI chip giant, which has been getting ready for the emergence of inference.
Inference has been such an intense area of investment that the trend has been hailed as “gold rush inference.” So ZML has competition like Basetenrecently valued at $13 billion; inferactfrom the creators of the open source project vllm; as well as RadixArkhe commercial company behind SGLang.
Both vLLM and SGLang partially compete with LLMD, but Morin’s ambitions for ZML cover a broader spectrum. “We’ve gotten to the point where we’re co-designing silicon,” he said. Additionally, he credited ZML’s small 20-person team for the reason the Paris-based startup has been able to move forward quickly, with more launches in the plans.
It also helped that this small team is well funded for its size. Thanks to his career as VP of engineering at Zenlythat Snapchat acquired by nine figures in 2017Morin raised $20 million from venture firms including Harry Stebbings’ 20VC, >commit, AALVC, Drysdale Ventures, Xavier Niel’s Kima Ventures, Kindred Capital, LocalGlobe and Puzzle Ventures.
Unlike the first public ZML project, the inference-centric approach Machine learning framework launched in 2024 and updated in marchZML/LLMD is not open source. But it is launched as a free product with the aim of learning about its use. “I prefer to measure and [then generate revenue] where it is most effective without stupidly hindering my growth because I have been too greedy from the beginning,” Morin said.
It is too early to know when ZML/LLMD could become a paid product and what its adoption will look like. But the startup’s cap table confirms that other founders are paying attention, including Dagger and Docker founder Solomon Hykes, Clément Delangue and Julien Chaumond of Hugging Face, as well as LeCun, now with AMI Laboratories. This also reinforces the argument that European AI startups now you can build from home. “I couldn’t do ZML anywhere else but Paris,” Morin said.
When you purchase through links in our articles, we may earn a small commission. This does not affect our editorial independence.