Foreword
Read little, type a lot. This book prepares you for a role as an Applied AI Engineer / SRE at an LLM provider: Linux and networking from the bottom up, your own tools in Python and Go, Mistral models on a VM and in Kubernetes, observability, on-call, security – and finally operations on-prem at the customer site.
The training rules
- Type, don’t copy. Run every command yourself once.
- Compare with Expected. After every drill you will find what you should see. If it differs, that is where the real lesson is.
- Pass the checkpoint. Only move on when the check script shows nothing but ✔.
- Solve the challenge on your own. First the hint, then the solution.
- Explain out loud. Present the Interview Note freely once in two minutes, as if an interviewer were sitting across from you.
- Lab values stay in the lab.
lab-key-acme,lab-key-globexand all passwords are play money. Real Mistral keys only viaMISTRAL_API_KEY, never in code.
Structure of every training day
| Section | Purpose |
|---|---|
| Briefing | The bare essentials in a few bullet points |
| Drills | Commands with expected output |
| Checkpoint | Check script: did everything work? |
| Challenge | A task without instructions |
| Interview Note | How you explain the topic in an interview |
| Enterprise Note | What is different in production and at the customer |
The training plan
| Week | Day | Unit |
|---|---|---|
| 1 – Foundations | 0 | Build the training ground |
| 1 | Linux internals: processes, /proc, cgroups v2, namespaces, systemd | |
| 2 | Network debugging: ss, dig, curl timings, tcpdump, TLS, MTU | |
| 3 | Python for SRE: build and test llmctl | |
| 4 | Go for SRE: build, test and ship llm-gateway | |
| 2 – LLMs & Kubernetes | 5 | Mistral models on a Linux VM |
| 6 | Multi-tenant Kubernetes: RBAC, quotas, NetworkPolicy, PSA | |
| 7 | Model serving on Kubernetes | |
| 3 – Observability & operations | 8 | Observability I: Prometheus, PromQL, Grafana |
| 9 | Observability II: Loki, OpenTelemetry, Tempo | |
| 10 | SLOs, alerting and on-call | |
| 11 | Incident response and post-mortem | |
| 4 – Automation, security, customer | 12 | Infrastructure as Code: OpenTofu and Ansible |
| 13 | Security: SDLC, SBOM, signatures, CVE response | |
| 14 | Hybrid and on-prem at the customer | |
| 15 | Competition: disaster drills | |
| Appendix | A | Cheat Sheet |
| B | Final Exam | |
| C | Interview preparation |
The training ground
Laptop (macOS/Linux) ~/sre-lab
┌───────────────────────────────────────────────────────────────────┐
│ docker · kind · kubectl · helm · go · python · tofu · ansible │
│ │
│ kind cluster "sre-lab" (1 control-plane + 2 worker) │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ ns llm-system: model server · llm-gateway │ │
│ │ ns monitoring: Prometheus · Grafana · Loki · Tempo │ │
│ │ ns tenant-acme: client apps (lab-key-acme) │ │
│ │ ns tenant-globex: client apps (lab-key-globex) │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │ HTTP /v1/chat/completions │
│ ▼ │
│ Linux VM "llm-vm" (multipass, Ubuntu 24.04) │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Ollama :11434 (CPU, Mistral model) │ │
│ │ vLLM :8000 (GPU only, e.g. cloud VM) │ │
│ │ systemd · cgroups · journald │ │
│ └──────────────────────────────────────────────────────────┘ │
└───────────────────────────────────────────────────────────────────┘
Job posting → training day
| Requirement | Days |
|---|---|
| Linux internals, network debugging, distributed systems | 1, 2, 15 |
| Python and Go: build, test and ship tools | 3, 4 |
| LLM/model serving | 5, 7 |
| Multi-tenant Kubernetes: namespaces, NetworkPolicy, RBAC | 6, 7 |
| Observability: Prometheus, Grafana, OTel, Loki, Tempo | 8, 9 |
| On-call, incident response, post-mortems, runbooks | 10, 11, 15 |
| IaC: Terraform/OpenTofu and Ansible | 12 |
| Security mindset: secure SDLC, CVE, supply chain | 13 |
| Hybrid/on-prem at the customer, customer communication | 11, 14 |
| Interviews and written communication | every Interview Note, Appendix C |
Tip: Tested with go 1.25 and 1.26, python 3.14, Ollama 0.35, kind 0.33, kubectl 1.37, helm 4.3, tofu 1.12 and Ubuntu 24.04. If a flag differs:
<tool> --helpand the official documentation.
The companion code
All files from this book are publicly available on GitHub: https://github.com/thomaszachmann/llm-plattformen-zum-anfassen. Day 0 shows you how to get them.
Let’s go.
Thomas Zachmann, October 2026