Article

Terraform Modules Done Properly: When to Abstract, When to Copy-Paste, and Why Your Module Library…

Terraform Modules Done Properly: When to Abstract, When to Copy-Paste, and Why Your Module Library Is Probably Too Clever

by Gary Worthington, More Than Monkeys

A few years ago I was brought in to look at a client’s Terraform estate, and the first thing I opened was a module called aws-base. It had 87 input variables. It created networking, compute, IAM, monitoring, and — for reasons nobody could reconstruct — an SQS queue that appeared in every environment whether you wanted one or not, like a regional development grant.

Every team in the company depended on it, yet nobody understood it. Changing anything in it required negotiating with four other teams, so nobody changed it, so every team had quietly started working around it, which meant the company now had five infrastructure patterns instead of one and a module whose main function was ceremonial.

Someone had built it with the best intentions. That’s how all of these start.

This is the third article in this series. The first covered how Terraform actually works and why the state file deserves your respect. The second argued that workspaces aren’t environments and showed the directory-per-environment layout, which quietly depends on a modules/ directory full of shared code. This one is about what goes in that directory — and, more importantly, what shouldn't.

A quick recap

The layout from last time: shared modules in modules/, and a thin root module per environment that calls them with that environment's values. The roots stay boring — backend config and module calls, nothing else. The modules hold the actual logic.

That structure only works if the modules themselves are good. A thin root calling a bloated, over-clever module hasn’t simplified anything; it’s just moved the mess somewhere with fewer witnesses.

So the real question is what makes a module good. The answer is mostly about restraint.

What a module actually is

Mechanically, a module is nothing special: any directory containing Terraform files. When you write resources in environments/prod/main.tf, you're already in a module — the root module. A module block just pulls in another directory and passes values across the boundary:

module "orders_service" {
source = "../../modules/ecs-service"
name = "orders"
environment = "prod"
desired_count = 6
}

The useful mental model is a function call. Variables are the arguments, outputs are the return values, and everything in between should be invisible to the caller. No reaching into a module’s internals, no module reading things it wasn’t passed — inputs in, outputs out, like a well-behaved function in any other language.

Which invites the same question you’d ask about any function: what is this abstraction for?

Modules encode decisions, not text

The common answer is “DRY — don’t repeat yourself”. It’s the wrong answer, or at least a dangerously incomplete one. Deduplicating text is the weakest reason to create a module, and modules created purely to deduplicate text are how you end up with aws-base.

The right reason: a module should encode a decision your organisation has made. “Our services run on ECS Fargate with these health checks, this log retention, alarms wired up like so, and tags that keep the finance team off our backs.” That’s a decision — several, in fact — and a module is how you write it down once and make the right thing the easy thing.

This framing does real work when you’re deciding what to build. A modules/ecs-service that bakes in your organisation's opinions about how a service should run: clearly earns its place. A modules/s3-bucket that wraps the aws_s3_bucket resource and re-exposes its arguments one by one: encodes nothing. It's aws_s3_bucket in a hat.

The wrapper module

That hat deserves its own section, because the single-resource wrapper is the most common bad module in existence and it always looks harmless in review.

A wrapper module adds a layer of indirection with no decisions inside it. The provider documentation no longer matches your code, because your variable names differ slightly from the resource’s arguments. When AWS ships a new bucket feature, the provider supports it immediately — but your wrapper doesn’t, so someone has to add variable number 23 and thread it through. Every consumer now depends on your wrapper’s release cycle for functionality the provider gave them for free.

And the payoff for all this is nothing. No decision was encoded. No complexity was hidden, because there wasn’t any. The wrapper exists because creating modules felt like good practice, the way buying a filing cabinet feels like being organised.

The rule: if a module contains one resource and no opinion, delete it and use the resource.

The god module

At the other end of the spectrum sits aws-base and its cousins — the module that provisions half the company's infrastructure through one enormous interface. God modules fail for the same reason god classes fail in software, but with a Terraform-specific twist that makes them worse: the blast radius problem.

Everything one module creates lives in one dependency graph, applied together, growing together. Teams that share a god module are coupled to each other’s changes whether they like it or not — one team’s urgent fix means a major-version bump for four teams who wanted nothing. So changes get batched, releases get scary, and eventually the module freezes solid while everyone tunnels around it.

There’s a size heuristic hiding in here. A good module covers things that change together for the same reason: one service and its alarms, one VPC and its subnets and route tables. The moment a module covers things that change at different speeds for different reasons — networking and compute and IAM — it will be permanently under-maintained at one end and dangerously churning at the other.

The rule: a module should have one reason to change. If you can’t finish the sentence “this module changes when ___” without using the word “or”, it’s two modules. Possibly five.

Copy-paste is not a moral failing

Between the wrapper and the god module lies the question every team argues about: when do you abstract?

Later than you think. My working rule is the old rule of three: write it once, write it again with a straight face, and only when the third copy appears do you consider a module — and even then, only if the three copies agree on what they’re doing. Two similar-looking blocks of Terraform that serve different purposes are not duplication; they’re coincidence, and abstracting a coincidence welds two unrelated things together at exactly the joint where they’ll need to differ next time around.

Engineers hate this advice, because copy-paste feels like failure and abstraction feels like craft. But a premature module is a bet that you already understand the pattern, placed at the moment you know least about it. The third usage is when the pattern shows you its true shape; which knobs actually vary, which were accidents of the first implementation.

Copy-paste is cheap to fix later. A wrong abstraction with six consumers is a renovation project.

What a good module looks like

Small surface, strong opinions. Here’s the shape, using the service module the environments have been calling since last article:

# modules/ecs-service/variables.tf
variable "name" { type = string }
variable "environment" { type = string }
variable "desired_count" { type = number }

variable "cpu" {
type = number
default = 512
validation {
condition = contains([256, 512, 1024, 2048], var.cpu)
error_message = "cpu must be a valid Fargate size."
}
}

Three required inputs, because only three things differ between services. Everything else — log retention, health check thresholds, deployment settings, the tagging scheme — is a default inside the module, because those are the decisions, and decisions shouldn’t be re-debatable at every call site. A consumer who needs to see them can read the module; a consumer who wants to override them, mostly, shouldn’t.

The validation block is worth the two minutes it takes. It converts “apply failed after four minutes with a Fargate error” into “plan failed in four seconds with a sentence” — which is the entire customer experience of your module, since your consumers’ first interaction with your opinions is an error message.

Notice also what’s absent: an environment-based conditional adjusting behaviour inside the module. The module takes values; the environment roots decide what the values are. The moment a module starts asking where am I?, you've rebuilt the conditional creep from part two, one layer down where it's harder to see.

Compose small modules; don’t configure big ones

When a service needs something extra — a queue, a cache, a bucket — the temptation is to add create_queue = true to the service module. Resist that too. Each toggle doubles the module's possible shapes; five toggles is thirty-two modules wearing a trenchcoat, and the test matrix to prove they all work never gets written.

The better pattern is composition in the root, where it’s visible:

module "orders_service" {
source = "../../modules/ecs-service"
name = "orders"
# ...
}

module "orders_queue" {
source = "../../modules/sqs-queue"
name = "orders-events"
consumer = module.orders_service.task_role_arn
}

The root module is the configuration language. Two module blocks side by side say “this service has a queue” more plainly than any boolean buried in a variables file — and the service module stays one thing that changes for one reason.

Versioning, and when to bother

In a monorepo with one team, relative paths (source = "../../modules/ecs-service") are correct, whatever the internet tells you. Module changes ride along with the roots in the same pull request, promotion is applying environments in order, and git is the audit trail. Version pinning here is ceremony without benefit.

Pinning earns its keep the day a second team consumes your modules — that’s when “the module changed under me” becomes possible, and tagged releases (?ref=v1.4.2 on a git source, or a private registry if you're fancy) become the contract between teams. Prod pins a version and upgrades deliberately; the module team can move without breaking anyone mid-sprint.

As for the community modules — the terraform-aws-modules collection on the public registry is well-maintained and the VPC one in particular encodes years of accumulated AWS folklore. Use them where the underlying AWS surface is genuinely fiddly. But read what they create before you apply, because you're adopting several hundred resourceful opinions at once, and check the changelog before every upgrade. A community module is a dependency like any other: excellent right up until the major version bump you didn't read about.

One structural rule while we’re here: keep nesting shallow. Roots call modules; modules may occasionally call a small helper module; and that’s the bottom of the staircase. Every level of nesting is a layer an on-call engineer has to dig through at speed, and by the third level you’re spelunking.

The uncomfortable truth

The client with the 87-variable module didn’t have a Terraform problem. They had a decision-avoidance problem wearing Terraform as a costume. Nobody had ever agreed what the company’s standard service looked like, so the module grew a variable for every disagreement — 87 arguments standing in for the ten decisions nobody wanted to chair a meeting about.

That’s the real job of a module library, and it’s why this article is short on syntax and long on restraint. Modules are where your organisation’s infrastructure decisions live. If the decisions haven’t been made, no amount of HCL will make them for you — you’ll just get very configurable indecision.

Make the decision. Encode it in a small module with a small interface. Copy-paste cheerfully until the pattern proves itself. And when someone proposes wrapping a single resource in a module for tidiness, ask them what decision it encodes, and watch the pull request quietly close itself.

Good abstractions are opinions that survived contact with three real uses. Everything else is filing cabinets.

Gary Worthington is a software engineer, delivery consultant, and fractional CTO who helps teams move fast, learn faster, and scale when it matters. He writes about modern engineering, product thinking, and helping teams ship things that matter.

Through his consultancy, More Than Monkeys, Gary helps startups and scaleups improve how they build software — from tech strategy and agile delivery to product validation and team development.

Visit morethanmonkeys.co.uk to learn how we can help you build better, faster.

Follow Gary on LinkedIn for practical insights into engineering leadership, agile delivery, and team performance.