AI
LLM Inference Is a Memory Problem:
Tonight our co-founder and CTO Jonas Vander speaks at Embedded Austria No. 7 in Vienna. His talk is titled “LLM Inference Is a Memory Problem”, and it covers the question that decides how AI models perform once they run on your own hardware.
The event
Embedded Austria brings together the Austrian embedded community, and this seventh edition is dedicated to AI in embedded development. It takes place on Thursday, 12 November 2026, from 18:00 at SBA Research, Floragasse 7, 1040 Vienna, and closes with open networking. Ticket sales have ended, so if you already have a ticket, come say hi to Jonas after his talk.
Why memory, not compute
When organisations bring language models onto their own servers, the first question is usually how many GPUs they need. The question that decides how fast answers come back and how many people can use a model at the same time is a different one: how much memory every model, every context and every parallel request claims, and how well that memory gets used.
Three things compete for it:
The model weights. A model has to sit in GPU memory before it answers anything, and larger models need more of it.
The context. Every conversation keeps a working memory of what has been said so far, and it grows with every document and every turn.
Parallel requests. Each person using the model at the same moment adds their own context, so a model that runs smoothly for one user can slow down for fifty.
Generating text is also limited by how fast data moves through memory, not only by how fast the chip can calculate. That is why two servers with similar compute can deliver very different results for the same model.
Who is speaking
Jonas has spent a decade in AI engineering and is the architect behind the Xinity Runtime, the open-source platform that lets organisations run AI models on their own servers. He works on this problem every day, from choosing which models fit on a customer’s hardware to keeping response times steady as more teams start using them.
The Xinity platform is open source on GitHub, if you want to see how it handles models on your own infrastructure before or after the talk.
Planning to run AI models on your own hardware? Book a discovery call and talk it through with our team.
AI declaration: This article was written with the support of AI tools and reviewed and edited by the Xinity team.