We Replaced Ollama With vLLM on CPU-Only Kubernetes — Here Is What Changed
· 9 min read
Most write-ups about vLLM versus Ollama assume you have a GPU. The benchmarks show impressive VRAM utilisation. The architecture diagrams include CUDA drivers. The recommendation — vLLM wins — comes with an implicit asterisk: assuming you have the hardware for it.
We did not. Our inference cluster is four ThinkPad laptops, each with an i7-8565U (4 cores, 8 threads, up to 4.6 GHz turbo), 16 GB of RAM, and no GPU of any kind. We ran Ollama first. Then we replaced it with vLLM. This is the honest account of that migration: what broke, what improved, and what the tradeoffs actually look like when you run LLM inference on commodity x86 CPUs.