Orchestrating Terraform, Ansible, and Helm for Edge Deployments

Edge deployments mean managing three layers at once: the site and its network connections, the machines in the building, and the applications running on top of them. Most teams end up with three tools to match, usually Terraform for infrastructure, Ansible for machine configuration, and Helm for applications.

Each tool is good at its own layer. The trouble starts where they meet. Terraform sets up a site, but Ansible needs to know what it created. Ansible prepares a machine, but Helm needs to know when it's ready.

This post covers how the three tools divide the work, where the handoffs between them fail, and how to roll out across hundreds of sites safely. 

Edge vs. Cloud Deployments

In the cloud, you ask for a server, and one appears a few seconds later. At the edge, someone drives to a building with a box in the trunk, and no software work starts until that visit happens.

The network is a suggestion. Sites go dark. A store loses connectivity during business hours. A factory floor sits behind a firewall nobody wants to touch. An update that hangs is worse than one that fails outright, because a failure tells you something.

No two sites are identical. Hardware arrives in generations, so you will have three in the field at once, two operating system versions, and one site running something nobody remembers installing. And nobody is there to reboot a machine or read an error off a screen. Every fix has to work unattended, or it isn't a fix.

Which leaves the tension that defines edge work: you need every site to behave the same way, and no two sites are the same.

Terraform, Ansible, and Helm

Terraform handles the site and its connections. It registers the location and creates the cloud-side resources the site talks to: networking, certificates, identity, and the registry the fleet is tracked in. Ansible handles the machines. 

Operating system, packages, disk layout, container runtime, and everything else that turns a box in a closet into a device ready to run software. Helm handles the applications on Kubernetes, upgrading them and rolling them back when an upgrade fails.

The rule that keeps this manageable: each tool owns one layer and never reaches into another. That rule gets broken constantly, always for a good reason at the time. 

Ansible writes Kubernetes manifests because the person who knew Helm was out that week. Terraform installs applications directly because it was already running. Both work fine until the fleet outgrows a single person's memory, and by then the shortcut is load-bearing.

Terraform, Ansible, and Helm Hand Off

This is the part nobody gets assigned. Each tool has documentation and a community behind it. The connections between them have neither, so they get built once, in a hurry, by whoever was on the ticket.

Getting the Machine to Answer

Terraform can't provision hardware. Something has to take a box from powered on to reachable before any playbook runs: a pre-baked image, cloud-init, PXE boot, or a vendor's zero-touch provisioning. Whatever you pick, it should end the same way. The machine calls home, presents an identity Terraform issued, and registers itself. Without it, someone on site reads an IP address down the phone.

From Terraform to Ansible

Ansible needs three things before it can start: the address to connect to, credentials to authenticate with, and enough detail about the host to decide which tasks apply.

A static inventory file works for the first 10 sites and then rots. Every new location means an edit, every decommissioned one leaves a stale entry, and nobody notices until a playbook fails against an address that no longer answers.

Build the inventory from what Terraform already knows instead. Tag-based cloud inventory plugins don't help here, because the box in the closet isn't a tagged instance. Point Ansible at the registry the site checked into, or generate the inventory from Terraform state as a build artifact. Either way, it stays current because it's derived rather than maintained. Group hosts by region, hardware generation, and rollout wave so playbooks can target subsets without a lookup table.

Credentials don't belong in that inventory. Issue short-lived, per-site credentials at runtime, tied to the identity the machine registered with. A shared key that reaches 900 sites can't be rotated without touching all of them.

From Ansible to Helm

A playbook exiting zero means the tasks ran. It doesn't mean the node can accept workloads. That gap is where "wait 60 seconds and hope" lives, and a fixed timer is wrong in both directions: too long on a healthy site, too short on the one with a failing disk.

Check real conditions instead.

kubectl wait --for=condition=Ready node/$NODE --timeout=300s

Then confirm the container runtime responds and storage is mounted and writable. On the Helm side, --wait --atomic --timeout turns a half-finished upgrade into an automatic rollback.

Make the playbooks safe to re-run while you're at it. Connectivity at the edge means some runs stop partway through, and the only sane recovery is running the same playbook again and having it continue. That means idempotent tasks and no steps that assume a clean starting state.

From Helm back to Terraform

Applications produce facts the infrastructure layer needs: an address to route traffic to, a certificate to publish, and a health endpoint for monitoring. Feeding those backward creates a circular dependency where Terraform waits on the workloads and the workloads wait on Terraform.

Split the Terraform work into two. The first stage creates what must exist before any workload runs: networking, identity, certificates, and storage. The second consumes what the workloads produce and configures routing and monitoring. One direction: Helm in between.

The common thread: the order the steps run in is a real dependency. If it only exists inside a shell script on somebody's laptop, it doesn't exist. Something has to hold that order and keep credentials out of the scripts. 

You get there by extending your existing continuous integration (CI) pipelines, building the coordination layer yourself, or running each tool as a separate unit in an orchestration platform. Spacelift, for one, models each tool as its own stack and wires them together with dependencies and output references, so Ansible starts only after Terraform succeeds, and only with the values Terraform produced.

Push or Pull?

Push means a central system sends updates to each site. It's simpler to build, and it stops working the moment a site is unreachable, which at the edge is most Tuesdays. Pull means each site checks in and fetches its own updates. It handles poor connectivity well and adds one more piece of software to maintain at every location, including the ones you can't reach.

That choice changes the last handoff. Under push, the sequence runs centrally, and Helm installs from your pipeline. Under pull, Terraform and Ansible still run centrally, but the Helm step becomes a chart version written to Git that an agent at each site reconciles on its own schedule. The readiness checks still apply; they run at the site instead.

A second decision sits alongside it. Treating the fleet as a single unit is straightforward until one bad change affects all of it at once. Treating each site as its own unit means a mistake stops at one site, and fleet-wide changes become a rollout you manage rather than one action across the whole fleet.

Lean toward pull and per-site units as the fleet grows, connectivity becomes less reliable, and the cost of a bad night rises. A dozen well-connected sites don't need what 900 stores across three continents do.

Roll Out Updates

Roll out in waves, starting with canary sites. Pick canaries that represent the awkward parts of the fleet, not the convenient ones. The site down the road with the newest hardware will tell you almost nothing.

Be honest about rollback. A helm rollback is quick and well understood. Undoing an operating system change on a machine nobody can reach is a different problem, and it deserves more caution before you ship than after.

Then there's configuration drift, which at the edge usually has a physical cause. A technician replaced a failed disk with whatever was in the van. Someone changed a network setting to get a checkout lane working before the doors opened, and it stayed that way. Decide in advance what corrects itself quietly and what wakes a person up.

Where to Start

Keep each tool inside its own layer, and write down what each stage hands to the next. That handoff is the actual engineering work, and it tends to live in nobody's documentation.

None of this gets easier by adding more tools. It gets easier when the connections between them are designed on purpose rather than grown.