Un modèle sort de l'entraînement à 28 Go. Tu le télécharges à 4.7 Go. Qui a fait la compression entre les deux — et pourquoi ?
Ce post répond à cette question, et explique comment les modèles open-source circulent depuis les labos de recherche jusqu'à ton cluster.
Un modèle 7B en FP32, c'est 28 Go de RAM. Mes ThinkPads en ont 16 à 32. Impossible de charger le modèle — sans parler de faire de l'inférence.
La quantization a résolu ce problème. Pas en sacrifiant la qualité — en changeant la façon dont les poids sont stockés.
The hardware was provisioned. MAAS had PXE-booted four ThinkPads and cloud-init had written the SSH keys and hostnames. The next question was: how do you actually get Kubernetes running on five laptops?
There are more ways to install Kubernetes than there are Kubernetes certification paths. I ended up with k3s. Here is why — and exactly what the installation looked like.
Running LLMs on bare-metal CPU hardware forces you to understand the numbers behind model files. This post documents how we reason about model size, quantization, and inference serving on minicloud — and the concrete change we made: replacing phi4-mini with Qwen 2.5 7B across all three Ollama instances.
Every time I updated the Homer dashboard config, I had to run kubectl rollout restart deployment/homer -n homer after ArgoCD finished syncing. Same for LiteLLM when routing changed. Same for Backstage after any catalog or proxy update. The pattern was identical every time: push to git, wait for ArgoCD sync, then manually trigger a pod restart.
That is an operational smell. If git is the only write path, the restart should be automatic too.
Most cloud platforms hide the infrastructure from you. MAAS provisioning, PXE boot sequences, NIC bonding, storage backends, certificate chains — all of it abstracted behind a few CLI flags or a dashboard. That abstraction is valuable in production, but it can also keep engineers at arm's length from the system they're supposed to understand deeply.
This project started from a simple question: what does it actually take to build a production-grade Kubernetes platform from scratch?