ZML releases LLMD, free software to cut AI inference costs

ZML, the French AI infrastructure startup publicly endorsed by Yann LeCun, released LLMD on July 8, a free product designed to speed up language model inference across a wide variety of chips, as reported by TechCrunch. The promise is straightforward: run the same model on hardware from different manufacturers without rewriting anything, and pay less for every token served.

What ZML is and what LLMD is

ZML is not starting from scratch. The company, founded by Steeve Morin, former CTO of Zenly, the social mapping app acquired by Snap, maintains a high performance inference framework written in Zig and published as open source on GitHub. Its core idea is to compile the model once and run it on NVIDIA GPUs, AMD accelerators or other silicon, instead of tying each deployment to a specific ecosystem.

LLMD is the production oriented piece. According to TechCrunch, it is free software aimed at serving language models at lower cost, building on that portability across chips. The endorsement from LeCun, a Turing Award winner and one of the most influential voices in AI research, works as a calling card in a market already saturated with inference engines. The article does not detail the exact license or the business model, so the distinction between free and open source remains subject to the fine print.

Why it matters

The cost of generative AI is no longer in training models but in serving them. Every query to a chatbot or an agent runs inference, and that bill grows with usage, not with the calendar. Today most serious deployments run on NVIDIA GPUs with software optimized for CUDA, which grants the manufacturer considerable pricing power. Tools like vLLM or TensorRT-LLM have improved performance a great deal, but the latter belongs to NVIDIA itself and the former was born inside that ecosystem.

A runtime that treats hardware as an interchangeable commodity attacks exactly that point. If LLMD delivers on its promise, a team could serve the same model on NVIDIA today, on AMD tomorrow, and on whichever accelerator offers the best price per token next month, negotiating with vendors instead of accepting rates. For the open weights ecosystem, with families like Llama, Mistral, Qwen or DeepSeek, where the user controls the weights and chooses where to run them, that flexibility translates into real money. The pressure also benefits those who never migrate: the mere existence of credible alternatives improves the negotiating position of any compute buyer.

There is also a European angle. ZML is French, like Mistral and like Hugging Face in its origins, and it reinforces the thesis that Europe can compete in the infrastructure layer even if it does not train frontier models. The fact that the product is free suggests the classic strategy of adoption first and monetization of support and enterprise services later, but that, again, is our reading and not information confirmed by the company.

Who this is useful for

Platform teams serving open weights models who want to stop depending on a single GPU vendor.
Companies with growing inference bills looking for leverage to negotiate hardware or switch clouds.
* European cloud providers that want to offer competitive inference on alternative silicon.

The fine print, as always, will be in the benchmarks. The history of portable runtimes is full of promises that left 20% or 30% of performance on the table compared with each manufacturer's native stacks, and that margin decides whether portability pays off. Our experience serving models in production makes us skeptical by default and curious by trade: LLMD goes straight onto the list of things we will test with our own metrics, because if performance parity is real, the economic argument stands on its own.