What Happens on terraform apply

Config, state, plan, providers

Follow one terraform apply end to end: which files Terraform reads, what init prepared, how the state is locked and refreshed, how the dependency graph becomes a plan, how provider plugins call the cloud APIs in parallel, and what is left behind when a step fails.

Overview

terraform apply makes real infrastructure match the configuration in a directory. It is a short command with a long path behind it: read every .tf file, lock the state, ask the cloud what really exists, work out the order of changes, show a plan, then call cloud APIs, often in parallel, and record each result.

One correction comes first, because the question is often asked as terraform apply -f file.tf. Terraform has no -f flag and never applies a single file. It loads every .tf and .tf.json file in the working directory as one configuration:

The interviewer listens for four things. Do you know what Terraform reads and what init set up before? Can you explain state: why it exists, why it is locked, and what refresh does with it? Can you say how the plan decides between update and replace, and in what order changes run? And do you know what happens when the run fails halfway, since nothing is rolled back?

Questions to ask first

Where is state kept? — A remote S3 backend shared by the team, with locking on
Who runs apply? — A CI pipeline applying a plan that was reviewed; engineers run plan locally
Which providers? — One cloud provider plus a few utility providers
How big is the configuration? — A few hundred resource instances in one root module (an assumption)
Has terraform init run? — Yes, and the dependency lock file is committed

What you will be able to do

Say exactly which files terraform apply reads, the real forms of the command, and what init prepared.
Explain state, the lock, refresh and drift, and recover from a stale lock.
Read a plan: every action symbol, why something is replaced instead of updated, and the order of changes.
Describe the parallel graph walk and the provider plugin calls behind each resource.
Explain what a failed apply leaves behind and how to continue safely, and run apply from CI on a reviewed plan.

Key concepts

Working directory
terraform init
Dependency lock file
Backend and state
State locking
Refresh and drift
Dependency graph (DAG)
Plan actions
create_before_destroy
Provider plugins
Parallelism
Tainted resources
Saved plans

Functional requirements

Load the whole configuration of the working directory: resources, data sources, variables, locals, modules, outputs
Run with the provider versions recorded in the dependency lock file
Allow only one operation at a time to change a given state
Detect changes made outside Terraform (drift) before planning
Show the exact set of changes and wait for approval, or apply a saved plan that was already approved
Create, update, replace and destroy resources in dependency order, in parallel where independent
Record every resource that exists after the run, including after a failure
Print outputs and release the lock when done

Non-functional requirements

Repeatable: applying the same configuration twice makes no changes the second time
Safe under concurrency: two runs never write the same state at once
Recoverable: a run that fails or is interrupted leaves state that matches what was created
Reviewable: what is applied in production is exactly the plan that was reviewed
Auditable: who changed what is visible in version control, the CI log and the state history
Least privilege: the credentials used by apply can change only what this configuration manages

Interview questions

Can you apply a single .tf file?
Why does Terraform need a state file?
What happens if two people run apply at the same time?
Why does the plan say "must be replaced" instead of updating?
Apply failed halfway. What state is the infrastructure in?
What does refresh do, and what is drift?
How does Terraform decide what runs in parallel?
How is terraform apply different from kubectl apply?