Why Not Just vLLM? An Honest Answer

We get this question in some technical evaluation, usually from the sharpest engineer in the room: "We could just run vLLM. Why would we pay you?"

It is a fair question, so it deserves a straight answer, not a feature table designed to make an open-source project look bad.

So here it is.

First: yes, vLLM is excellent

vLLM is one of the best inference engines available. It is fast, actively developed, widely deployed, and free. The same is true for other strong projects in the ecosystem. If your question is "can this serve tokens from a model on a GPU", the answer is yes, and it does it well.

If you are a research team, a startup without regulatory constraints, or an organization with a dedicated ML platform team, you may not need us. We would rather tell you that on our blog than have you find out after a sales call.

Our core engine is open source under Apache 2.0 for the same reason. You can take it, run it, and inspect it without ever talking to us. Some organizations do exactly that, and that is a fine outcome.

The question your auditor will ask is different

The engineer's question is "can we serve models". The organization's question, the one that actually decides whether an AI system reaches production in a bank, a hospital, or a public institution, is different:

Who has access to this system, and how is that access controlled? Where is the record of what was processed, by whom, and when? Who patches it when a vulnerability is published? Who is contractually liable when something breaks? Where is the data processing agreement? Who supports it at 8 a.m. before the board demo?

None of these are inference questions. All of them are production questions. And in a regulated environment, they are not optional paperwork around the system. They are the conditions under which the system is allowed to exist.

An inference engine, however good, does not answer them. It was never meant to. That is not a criticism of vLLM. It is a scope statement.

"Klaus maintains it" is not an audit answer

The realistic alternative to buying an inference platform is not vLLM out of the box. It is vLLM plus everything you build around it: authentication and role-based access control, request logging that satisfies an auditor rather than a debugger, model deployment and routing policy, update and patch processes, monitoring, documentation, and someone accountable for all of it.

That is a real project. Organizations with strong platform teams do it, and do it well. But it has two costs that rarely appear in the initial estimate.

The first is people. Building and operating this layer is realistically a significant part of one or two infrastructure engineers, ongoing, not once. In the DACH market, that is a six-figure annual commitment before the first business use case ships.

The second is auditability. Self-built infrastructure has to be defended in every audit, individually, by your team. When the auditor asks who maintains the inference layer, "our engineer set it up" starts a long conversation. A named vendor with a support contract, a documented patch process, and a signed DPA ends one.

This is the actual difference between an engine and a platform. Not throughput. Accountability.

What Xinity Runtime adds, concretely

Xinity Runtime is built for exactly this gap. On top of local, OpenAI-compatible inference on your own hardware, the platform layer provides role-based access control, audit trails on inference requests, deployment and routing of open-weight models including European sovereign models, multi-node operation, and a vendor in Vienna who signs a contract, provides support, and is liable for the software.

The open-source core stays inspectable. Your security team can read the code instead of trusting a black box. The commercial layer adds what your compliance team, your auditor, and your works council need to say yes.

In other words: engineers adopt the engine. Organizations procure the platform. Both are the same stack.

When you should not buy from us

To keep this honest, the cases where vLLM alone, or your own build, is the right call:

You have no regulatory constraints and no audit obligations. You have a platform team that builds and operates internal infrastructure as its job, and this is one more service in their portfolio. You are experimenting, not deploying, and production questions are a problem for next year.

In those cases, use the open-source ecosystem, including our core if it fits. Come back when the audit is scheduled.

The short version

vLLM answers "can we run models on our GPUs." Xinity Runtime answers "are we allowed to put this in production, and can we prove it."

For regulated organizations, the second question is the one that has been blocking AI adoption. It is also the one we built the company around.

Xinity Runtime turns your own servers into a compliant AI inference platform, open source at its core and deployable behind your firewall. If you want to see what that looks like for your infrastructure: Book a demo or start with the open source.

This post was drafted with AI assistance and reviewed by the Xinity team before publication.