Terraform Drift Detection: How to Find and Safely Reconcile Infrastructure Changes

Overview

Infrastructure drift occurs when a resource changes outside the normal Terraform workflow, or when Terraform configuration and state no longer describe the environment accurately. A change made in a cloud console is one common example, but drift can also result from an emergency command, an automated system, a provider behavior change, or a resource that was created without being added to the Terraform project.

It is useful to distinguish three related conditions:

• Configuration versus live infrastructure: the code declares one result, while the provider reports another.
• State versus live infrastructure: Terraform state contains an older view of a resource or no longer has the correct association.
• Configuration versus state: the code or state has changed even though the live resource may not have changed yet.

A normal Terraform plan is the primary inspection tool. Terraform refreshes its view of managed resources and compares that view with the configuration, then proposes actions. A plan is evidence for review, not automatic permission to apply. For a state-focused investigation, use a refresh-only plan:

terraform plan -refresh-only

This can help separate “the live resource changed” from “the configuration now requests a change.” Run commands from the correct workspace, with the intended variable files and provider versions. A remote backend with locking is strongly preferable for team workflows; see Terraform State Management: Remote Backends, Locking, and Safe Team Workflows before changing shared state.

Checklist by scenario

Scenario 1: A scheduled plan reports changes

1. Record the workspace, account or subscription, region, commit, Terraform version, provider versions, and variable sources.
2. Run the plan again from a controlled environment to rule out a transient credential, workspace, or provider problem.
3. Read the plan by resource address. Note whether each item is an update, replacement, creation, or destruction.
4. Compare the proposed attributes with the last approved change and the live resource.
5. Classify each difference as intentional, accidental, emergency-related, or unexplained.
6. Do not apply until the owner and impact of every consequential change are understood.

For CI, a plan can provide a useful signal without applying infrastructure. The following pattern uses Terraform’s detailed exit codes; verify the behavior in the Terraform version and wrapper used by your pipeline:

terraform init -input=false
terraform plan -input=false -detailed-exitcode -out=tfplan

# Exit code 0: no changes
# Exit code 1: plan error
# Exit code 2: changes detected

Store the plan output and review metadata according to your security rules. Plans can expose sensitive values or operational details, so avoid publishing them broadly in logs or pull-request comments.

Scenario 2: Someone made an intentional console change

1. Confirm who approved the change, why it was needed, and whether it is still required.
2. Decide whether Terraform should become the source of truth for the new setting.
3. If yes, update the configuration to represent the live intent, then run a plan.
4. If no, determine whether applying the existing configuration is safe and expected.
5. Document the decision in the change record and remove the temporary exception if one was created.

For example, if an engineer temporarily changes a security group rule during an incident, do not blindly copy that rule into code. First establish whether it is still necessary, whether it creates excessive access, and whether a narrower rule or a different recovery procedure is appropriate. IAM and access changes deserve an especially careful review; the Cloud IAM Best Practices guide provides a useful companion checklist.

Scenario 3: A change was accidental

Use the plan to determine whether restoring the declared configuration will update the resource or replace it. Check dependencies, maintenance impact, data retention, and provider-specific replacement behavior. If the resource is critical, create a recovery plan and obtain approval before applying. Do not edit state merely to make the plan look clean; state edits can hide the underlying problem.

Scenario 4: Terraform lost track of a resource

If a resource exists in the provider but is absent from state, first confirm that it belongs in this Terraform project and workspace. If it should be managed, use the appropriate import workflow, then adjust the configuration until a plan reflects the desired settings. If a resource address changed because of a module or naming refactor, investigate moved blocks or a carefully reviewed state move rather than recreating the resource. Provider upgrades can alter schemas and behavior, so review the Terraform Provider Upgrade Workflow before combining an upgrade with drift remediation.

What to double-check

• Workspace and account: confirm that credentials, subscription or account, region, and workspace all point to the intended environment.
• Variables and overlays: use the same variable files, environment variables, and generated inputs used by the deployment workflow.
• Ignored attributes: review ignore_changes. It may intentionally suppress a difference, but it can also conceal an unmanaged change.
• Computed values: some attributes are provider-populated or calculated. A displayed difference is not automatically a defect.
• Replacement markers: treat any “must be replaced” action as a separate risk review, especially for databases, networking, and persistent storage.
• Dependencies: inspect related resources, data sources, modules, and outputs before changing one object in isolation.
• Secrets: do not place credentials in configuration, plan artifacts, issue comments, or ad hoc shell history. For secret-storage design, compare the approaches in Cloud Secrets Management Compared.
• Cost and ownership: a drift fix can change resource size, retention, logging, or network usage. Check allocation tags and ownership records before applying; the cloud cost allocation guide can support that review.

Simple approval record

Keep a short record for each material drift event:

Environment and workspace:
Detected by and date:
Resource addresses:
Observed live change:
Likely cause:
Classification: intentional / accidental / emergency / unknown
Decision: update code / apply existing code / import / state move / defer
Risk and rollback plan:
Approver:
Follow-up owner and due date:

Common mistakes

• Applying the first plan: a plan can contain unrelated changes caused by stale variables, a wrong workspace, or a provider change.
• Using refresh-only as remediation: refreshing state records what Terraform sees; it does not necessarily make configuration and infrastructure agree.
• Deleting and recreating to clear drift: replacement may cause downtime, data loss, or a new identifier. Investigate import, configuration changes, or a state move first.
• Editing state without a backup and approval: state operations should be exceptional, deliberate, and performed with locking and recovery steps.
• Silencing every difference with ignore_changes: suppression is appropriate only when ownership and lifecycle are explicit.
• Mixing upgrades with reconciliation: change one major variable at a time where practical. A provider upgrade and drift fix in the same plan can make diagnosis harder.
• Ignoring the reason for manual changes: repeated console edits often indicate missing automation, an unclear ownership boundary, or an emergency process that needs improvement.

When to revisit

Run drift detection on a schedule appropriate to the risk and change rate of each environment, and always review it after an emergency intervention, console change, account migration, provider upgrade, module refactor, or incident. Before a seasonal planning cycle, confirm that scheduled plans, credentials, backends, notification routes, and approval rules still work. Recheck the workflow whenever Terraform, providers, CI runners, policy checks, or cloud account structures change.

A practical recurring routine is:

1. Run a non-applying plan in each managed environment.
2. Route changes to the owning team with resource addresses and a plan artifact handled securely.
3. Classify and record every material difference.
4. Resolve intentional drift by updating code or formally documenting an ownership boundary.
5. Resolve accidental drift through an approved plan, import, or state operation.
6. Review unresolved items at the next infrastructure or reliability meeting.

Drift detection is most effective when it becomes a feedback loop rather than a one-time cleanup. Pair it with a controlled Terraform CI/CD workflow, consistent monitoring, and clear ownership. Before your next plan, use this article as the checklist: verify context, inspect the proposed actions, classify the cause, obtain approval, reconcile deliberately, and record what changed.