Managing Terraform State in Teams: Remote Backends and Locking

This is the fourth post in our Terraform fundamentals series. It builds directly on the state concepts introduced in Terraform 101.

We flagged this problem back in the first post in this series and deliberately left it unresolved: local state — a terraform.tfstate file sitting on whoever's laptop ran apply first — works fine right up until a second person needs to run Terraform against the same infrastructure. This post is the fix, and because state is genuinely the highest-stakes part of running Terraform at any real scale, it's worth going deeper here than in the rest of this series — including how to actually inspect, repair, and migrate state when something goes wrong, not just how to configure a backend correctly the first time.

Why local state breaks down with more than one person

Picture two engineers, both with a local clone of the same Terraform configuration, both with permission to run apply. Engineer A runs it Monday morning and provisions a new subnet. Their local terraform.tfstate now reflects that subnet's existence. Engineer B, who hasn't pulled a state file that was never committed to version control in the first place, runs apply Monday afternoon — and as far as their state file is concerned, that subnet doesn't exist. Terraform tries to create it again.

Best case, this fails loudly with a naming collision. Worse case, it partially succeeds, and now two different state files each have a partially-incompatible picture of the same real infrastructure. This isn't a hypothetical edge case — it's the default outcome of local state the moment more than one person, or more than one CI job, touches the same configuration.

The fix: a remote backend

A backend tells Terraform where to store state — instead of a local file, a shared location every team member and every CI job reads from and writes to.

terraform {
  backend "s3" {
    bucket         = "novuspark-terraform-state"
    key            = "web-app/production/terraform.tfstate"
    region         = "eu-west-2"
    dynamodb_table = "terraform-state-lock"
    encrypt        = true
  }
}

Run terraform init after adding this block to an existing local-state configuration, and Terraform offers to migrate your existing state into the new backend automatically — it doesn't need to start from scratch.

A few things in that block are doing real work, not just configuration boilerplate:

• encrypt = true ensures the state file — which, as covered in the first post, can contain plaintext secrets — is encrypted at rest in S3.
• key is the path within the bucket. The pattern <project>/<environment>/terraform.tfstate is worth adopting from day one: it's what lets one S3 bucket safely hold state for many projects and environments without collisions.
• dynamodb_table is the part most tutorials skip, and it's the single most important line in this block.

State locking: the problem a backend alone doesn't solve

Moving state to S3 solves the "two people, two different pictures of reality" problem. It does not, by itself, solve a narrower but still serious problem: two people running apply at the exact same moment, both reading the same state, both computing a plan, both writing back — with the second write silently clobbering the first.

That's what the DynamoDB table is for. Before writing to state, Terraform acquires a lock by writing a row to that table; a second apply attempting to run concurrently sees the lock and blocks — or fails clearly — instead of racing.

resource "aws_dynamodb_table" "terraform_lock" {
  name         = "terraform-state-lock"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID"
 
  attribute {
    name = "LockID"
    type = "S"
  }
}

(This table itself is usually provisioned once, by hand or via a small separate "bootstrap" Terraform configuration with its own local state — a rare, deliberate exception to "always use a remote backend," since this resource has to exist before your main configuration's remote backend can use it.)

With locking in place, a second apply attempted mid-run gets a clear, immediate error instead of a silent race:

Error: Error acquiring the state lock

Lock Info:
  ID:        7f3a2b91-...
  Path:      novuspark-terraform-state/web-app/production/terraform.tfstate
  Operation: OperationTypeApply
  Who:       jane@novuspark.com

That error is the system working correctly, not a malfunction — it's telling you exactly who's currently holding the lock and what they're doing.

Inspecting state directly: the commands most engineers never learn

Beyond plan and apply, Terraform ships a set of subcommands purpose-built for reading and carefully modifying state directly — genuinely useful once a configuration has been running in production for a while, and worth knowing before you actually need them under pressure.

terraform state list

aws_instance.web
aws_dynamodb_table.terraform_lock
module.vpc.aws_vpc.main
module.vpc.aws_subnet.private[0]

state list prints every resource Terraform currently tracks, including resources nested inside modules — genuinely useful for answering "does this configuration actually manage that thing," without reading through every .tf file to check.

terraform state show aws_instance.web

state show prints every attribute Terraform has recorded for one specific resource — effectively a formatted view into the same JSON covered in the first post, without needing to open the state file directly.

terraform state mv aws_instance.web aws_instance.web_server

state mv renames a resource within state, without touching the real infrastructure at all. This matters specifically because renaming a resource block in your .tf file, by itself, tells Terraform "the old resource no longer exists and a new one needs to be created" — state mv is what tells Terraform "this is the same real resource, just tracked under a new name," avoiding an unnecessary and potentially destructive destroy-and-recreate.

terraform state rm aws_instance.legacy_bastion

state rm removes a resource from state without destroying the real infrastructure — Terraform simply forgets about it. This is the correct move when a resource is being intentionally handed off to be managed some other way (a different Terraform configuration, or manually), not when you actually want it deleted.

Importing existing infrastructure into state

Real organizations rarely start every piece of infrastructure from a Terraform apply. Something provisioned manually, or by a different tool, months before Terraform entered the picture, can be brought under Terraform's management with import:

terraform import aws_instance.web i-0abcd1234efgh5678

This adds the existing EC2 instance to state, associating it with the aws_instance.web resource block — but critically, it does not generate the .tf configuration for you. You still need to write a resource block whose arguments match the real instance's actual configuration closely enough that the next terraform plan doesn't propose a pile of unwanted changes. Getting this wrong is a common source of "I imported this and now plan wants to change twelve things I never touched" — worth writing the resource block carefully, running plan immediately after import, and reconciling any unexpected diff before moving on.

Recovering from a corrupted or lost state file

This is the scenario worth understanding before it happens, not while it's happening. A state file can become corrupted (a failed write, a bad manual edit) or simply lost (an accidentally deleted S3 object, absent versioning). The recovery path depends entirely on preparation done in advance:

• Enable S3 versioning on the state bucket. This is the single highest-leverage preparation for state recovery — a corrupted or overwritten state file is one aws s3api list-object-versions and a version-restore away from being fixed, rather than a genuine emergency.
• terraform state pull / terraform state push let you download the current remote state to a local file, inspect or carefully hand-edit it, and push a corrected version back — a last resort, appropriate only when you understand exactly what's wrong and exactly what the corrected JSON should look like.
• As a genuine last resort, terraform import every resource again against a fresh, empty state file. This works, but it's slow and error-prone for anything beyond a handful of resources — which is precisely the argument for having S3 versioning enabled long before you'd ever need it.

What this means for how a team actually works

Once state is remote and locked, a few practices become both possible and necessary:

• CI can run Terraform safely. Local state made CI genuinely dangerous — a pipeline run and a developer's local apply would fight over the same file with no coordination at all. Remote state with locking is what makes "Terraform runs in CI, humans don't apply from their laptops" a safe default instead of a risk.
• State access itself becomes something to control. Whoever can read the state bucket can potentially read every secret embedded in it. IAM policies on the state bucket deserve the same scrutiny as IAM policies on the infrastructure the state describes — a narrowly-scoped bucket policy restricting read access to the specific roles that genuinely need it is not optional hardening, it's the baseline.
• One state file per environment, not one giant shared file. Splitting dev, staging, and production into separate state files (via separate key values, as shown above) means a mistake in one environment's plan can't accidentally touch another's — the blast radius of any single apply is contained to the state file it's actually locking.
• Terraform Cloud and Enterprise offer remote state as a managed service, with locking, encryption, versioning, and access control built in rather than assembled from S3 and DynamoDB by hand — a genuinely reasonable alternative to the self-managed backend shown in this post, particularly for teams that don't already have strong AWS-native operational conventions to lean on.

What to actually remember from this post

• Local state is a single-person, single-laptop tool — it stops being safe the moment a second person or a CI job needs to run against the same infrastructure.
• A remote backend (S3, Terraform Cloud, or similar) gives everyone the same, current picture of what exists.
• Locking (DynamoDB, or a backend's built-in equivalent) is what prevents two concurrent applies from corrupting state — a backend without locking only solves half the problem.
• state list, state show, state mv, and state rm are real operational tools, not advanced trivia — worth knowing before an incident forces you to learn them under pressure.
• S3 versioning on the state bucket is the cheapest insurance a team can buy against a corrupted or accidentally-deleted state file.
• Encrypt state at rest, always — it frequently contains secrets whether you intended it to or not.

Next in the series: Terraform Workspaces and Multi-Environment Strategies, where we look at workspaces as one specific — and sometimes over-used — answer to managing dev/staging/production with the same configuration.