Under the Hood: Kubernetes QoS Classes

It is 3 a.m. and a node in your production cluster runs low on memory. When the dust settles, your payment service has been killed and restarted twice. The nightly batch job on the same node, the one nobody would miss for an hour, sailed through untouched. Nothing in the dashboards says why, and nobody made that choice on purpose. Except somebody did. Months earlier, someone wrote requests and limits into two manifests. The batch job got requests… Read More

Explained: Open Policy Agent

A new engineer asks who is allowed to delete an invoice. You say you will find out, which is the first sign of trouble. The answer lives in four places. There is a role check in the API gateway config. There is an if user.role == "admin" in the invoices service. There is a slightly different check in the nightly cleanup job, written a year later by someone who did not know about the first two. And there is a comment in the billing servi… Read More

How to Write a Kubernetes Operator

The previous post explained what a Kubernetes Operator is: a controller that watches a custom resource and continuously reconciles the actual cluster state toward the desired state. That explanation is useful. This one is different. This one ends with a working Operator. We're going to build one end-to-end: define a custom resource, write the reconciliation logic in Go, see the equivalent in Java, wire up the RBAC, and deploy it to a cluster.… Read More

Explained: Kubernetes Operators

You're deploying Kafka to Kubernetes. You write a Deployment for the brokers, a StatefulSet for ZooKeeper, ConfigMap s for configuration, Services to expose everything, and a handful of PersistentVolumeClaim s for storage. The cluster comes up. Then a broker dies and Kubernetes restarts it, but it comes back with the wrong broker ID and can't rejoin the cluster. Kubernetes did what it was told: restart the container. But it had no idea wha… Read More