zkdefi · notes

The Terraform-Kubernetes-Ansible-PKI-ACME pipe that started this whole thing

Five tools, in the order they touch the metal: Terraform reserves the resources, Kubernetes orchestrates the workloads, Ansible configures what Terraform can't, PKI roots trust into a name hierarchy, ACME automates the leaf certificates that make that trust visible to browsers. Each is the right answer to a different question. Together they're a pipe — provisioning, runtime, configuration, identity, trust — that I built in late 2021 as a personal exercise in "if I had to redeploy my whole infrastructure from a cold start, what would the script look like?"

The repository is small. The README is a single line. The actual content is a Terraform module, a small set of Ansible playbooks, a Helm chart or two, and a couple of bash scripts that thread them together. It's not impressive on its own. What's interesting in retrospect is the shape. I keep finding myself, four years later, writing things that look like specialized branches of this same pipe.

The reason it was worth building in 2021, and the reason it still pays compound interest: each of those five tools has a different theory of what change means. Terraform thinks change is "compare current state to desired state, apply the diff." Kubernetes thinks change is "watch the state and reconcile toward declared spec." Ansible thinks change is "run the script and hope it's idempotent." PKI thinks change is "issue a new certificate; the old one expires." ACME thinks change is "the issuer pushes a renewal at us before expiry, automatically." Reconciling those five theories of change is most of what makes self-hosted infrastructure hard. The pipe is one possible reconciliation.

The pipe also encodes one of the more durable lessons of operations: PKI is the load-bearing primitive, and ACME is what makes it bearable. Without PKI you have ad-hoc trust everywhere. Without ACME you have ad-hoc certificate rotation — meaning, in practice, certificates that expire on a holiday weekend. The combination is what makes the whole thing run on autopilot.

Looking at the commit history now, this is the project I keep cannibalizing. The Caddy + Forgejo + zkdefi.org stack that hosts this blog inherits the ACME logic. The PKI mindset shows up in every cryptography project I've done since. The Terraform + Ansible idiom is what I reached for again recently for the very server you're reading this post on.

What becomes possible: when "I need to stand up infrastructure" stops being a project and starts being a routine, you can do work that requires having infrastructure already standing.

#infrastructure #terraform #kubernetes #ansible #pki #acme #prehistory