Infrastructure as Code (IaC)

Definition

Infrastructure as Code — universally shortened to IaC — is the practice of describing infrastructure in version-controlled text files and applying those definitions through automated tooling instead of clicking through a cloud console. Terraform, Pulumi, AWS CloudFormation, Azure Bicep and CDK are the common instruments; the practice matters far more than the tool. Done well, IaC gives teams reproducible environments, auditable change history and disaster-recovery confidence. Done badly, it gives them a new class of production incidents.

Why It Matters

Before IaC, environments drifted. The staging cluster was created in 2019 by an engineer who has since left, using a console configuration nobody quite remembers, patched dozens of times, and it now differs from production in ways nobody has documented. When staging tests pass and production breaks, the drift is usually the cause. IaC makes the environment an artefact — reviewed, versioned, deployed and destroyed the same way as application code. It turns "we cannot rebuild production from scratch" into "we rebuild production from scratch every Tuesday morning in the DR account."

What Belongs Under IaC

• Compute — virtual machines, container clusters, serverless functions.
• Networking — VPCs, subnets, security groups, load balancers, DNS records.
• Databases and stateful services — instance definitions, backups, replicas.
• Identity and access — IAM roles, service accounts, RBAC bindings.
• Observability — dashboards, alerts, log routing, tracing configuration.
• Cost controls — budget alerts, quota limits, tagging policies.

Application deployments themselves usually run through a separate CD pipeline. The IaC layer is the platform underneath.

Real-World Example

A fintech team ran three environments — development, staging and production — all provisioned through the AWS console over four years. Each had accumulated bespoke tweaks. A regulator asked for a walkthrough of how production could be rebuilt after a catastrophic incident. The answer was "we would call the two senior engineers who remember most of it." The regulator was not satisfied. The team spent six months porting the estate to Terraform, splitting modules per service, and standing up an isolated DR account rebuilt from the same modules on a weekly schedule. The first rebuild took three days and revealed 47 undocumented differences. By month three the rebuild ran clean in 90 minutes and the differences were down to two, both intentional. When the next regulatory audit came, the walkthrough took 30 minutes and closed with no findings. The team also caught two production misconfigurations during the porting exercise that had been silently increasing bill and risk for over a year.

Practical Lessons Learned

• Modularise by service, not by resource type. A module per service is understandable; a module of "all our S3 buckets" makes changes ripple across unrelated teams.
• State is a shared production artefact. Terraform state, Pulumi state — treat it with the same care as a production database. Locked, backed up, access-controlled.
• Plan before apply. The plan output is the change review; the apply is the execution. Reviewing an apply after the fact is not review.
• Destroy environments regularly. A DR account rebuilt from the same modules once a week is the only real test of your IaC.
• Do not manually edit resources under IaC control. Drift undermines every guarantee the practice provides.

The Drift Problem

The moment somebody clicks in the console to fix an incident, the console reality and the IaC reality diverge. On the next apply the change is silently reverted, an outage follows, and the team decides IaC is dangerous. It is not — the manual edit was dangerous. Mature teams accept that emergency console edits will happen and treat them as a special kind of debt: recorded in the incident channel, reflected back into the IaC within days, and closed out with a drift-check job that catches any that were forgotten.

Expert Tips

• Version modules like libraries. Semver, changelog, pinned versions in downstream stacks. Consumers upgrade deliberately.
• Run policy-as-code alongside IaC. Open Policy Agent, Sentinel, or built-in cloud guardrails catch dangerous changes in the plan, before apply.
• Tag every resource with owner and cost centre at creation. Retrofitting tags across a live estate is thankless work.
• Keep secrets out of the IaC repo. Reference them from a secrets manager; the repo contains the reference, never the value.
• Store the state files in the account that owns the resources. Cross-account state ownership creates recovery scenarios nobody wants to rehearse.

Common Mistakes

• Manual console edits that quietly diverge from the definitions.
• One monolithic module that requires every team to coordinate every change.
• State stored in a personal S3 bucket with no backups.
• Secrets committed to the repo — sometimes in git history long after they were removed from the current file.
• No environment rebuild rehearsal; disaster recovery is theoretical.
• Skipping the plan review — approving the apply blindly because "the pipeline is green."

Key Takeaways

• IaC turns infrastructure into a reviewed, versioned, reproducible artefact.
• Modularise by service; version modules like libraries; store state like a production database.
• Rebuild an isolated environment regularly — it is the only real test of your definitions.
• Drift is the enemy; catch it with automated checks and reflect emergency edits back into the code.
• Policy-as-code catches dangerous changes at plan time, before they reach production.

Related Concepts

Interlocks with Continuous Integration, Blue-Green Deployment, Incident Management, Observability and Chaos Engineering. Reference modules and policy examples at PMMilestone.org.