Background
Terraform, an open-source Infrastructure as Code (IaC) tool developed by HashiCorp, is extensively used in Oracle Cloud Infrastructure (OCI) to automate the provisioning and management of cloud resources. It enables users to define infrastructure configurations in a declarative language, allowing for consistent and repeatable deployments of OCI resources such as compute instances, networking components, and storage.
A common concern for users of Terraform is the management of state when Terraform is used in a multi-user environment. Terraform maintains a local representation of a cloud deployment in a terraform.tfstate file, and reconciles any create, update, or delete requests against this state. Multiple users, working from their own disparate client environments, should have the ability to collaborate on the maintenance of a cloud environment without overwriting each other’s changes.
To address this concern, HashiCorp added the concept of remote state management to Terraform whereby state files could be stored remotely inside a cloud service provider’s (CSP) object storage and accessed by all collaborators.
OCI supports this capability in a couple of ways. First, OCI has long supported use of OCI Object Storage as a backend for Terraform via the S3-compatible backend.
Second, OCI offers a fully managed & free Terraform-as-a-Service tool called Resource Manager which manages multi-user state remotely while also providing other benefits like drift detection, environment introspection, and identity & access (IAM) integration. While Resource Manager provides a complete solution, many of our multi-cloud customers prefer to use native Terraform APIs and tools rather than updating their IaC code for each CSP.
Locking & Long-Lived Key Challenge
While the S3-compatible backend solves the remote state problem, it does not natively handle race conditions where two clients could potentially write state at the same time, thereby corrupting the remote state file or resulting in a non-deterministic end state. HashiCorp solved for this by introducing the concept of state locking and many CSPs created solutions that leveraged databases (i.e. CosmosDB or DynamoDB) to handle the locking. However, the S3-compatible-backend, which for years was the only option for OCI, is a stateless implementation and does not provide state locking.
In addition, while the OCI Terraform provider supports the use of short-lived tokens for authentication, the S3-compatible backend requires persistent, long-lived keys for authentication. OCI customers whose security policies & procedures prohibit distribution of persistent keys to employee laptops, especially without automated solutions for key rotation or revocation, found this aspect of the backend particularly problematic.
Welcome the New OCI Native Backend
OCI is pleased to introduce, in partnership with our friends at HashiCorp, an OCI native backend, that removes the dependency on the S3-compatible backend and natively addresses the state locking and long-lived key challenge. The new backend is supported starting with version 1.12 of Terraform and the documentation for the new backend can be found here.
State Locking
Rather than using a cloud database to manage locking, the backend leverages OCI object storage’s support for optimistic concurrency control by enabling conditional operations that prevent unintended race conditions.
When a client initiates a Terraform plan/apply/destroy operation, the If-None-Match: * HTTP header is used by the backend to place a lock file in the remote state bucket while enforcing a first-writer-wins model; the request succeeds only if no lock object already exists in the bucket while subsequent concurrent requests fail with a 412 Precondition Failed if the lock already exists.
This logic ensures atomicity of operations and a client attempting to perform an operation while another is in-process will receive an “Error acquiring the state lock” message and be prompted to retry.
Token-Based Authentication
The OCI backend supports all of the same authentication protocols that are supported by the OCI Terraform provider. These include:
- API keys
- Instance Principal
- Resource Principal
- OKE Workload Identity
- Security Tokens
This means that customers can standardize their security approach across OCI Terraform operations using whatever authentication method is preferred.
For those looking to do token authentication across the board, they can today leverage OCI’s token-based authentication to create a short-lived session token for user access. In the near future, even more possibilities will exist with the upcoming release of the OCI Token Exchange Service.
Working with the OCI Backend
Make sure that you’ve installed version 1.12 of Terraform (or later) and, in your local Terraform directory, initialized the backend by issuing a “terraform init” command. In addition, you should have the latest version of the OCI command line interface (CLI) installed as well.
To enable a remote backend for a Terraform project, simply add a backend.tf file (or add the following code block into any .tf file in your directory)
terraform {
backend "oci" {
bucket = "tf-remote-state"
namespace = "foobar"
}
}
The namespace should correspond to your OCI tenancy’s object storage namespace and the bucket name should correspond to a new bucket that you’ve already created for this purpose with that name. You can now initialize the backend by issuing the terraform init command in your working directory. If you have a local terraform.state or have a previous remote state configured in the same bucket (from the S3-compatible backend) then be sure to provide the -migrate-state flag to your init command.
$> terraform init Initializing the backend... Successfully configured the backend "oci"! Terraform will automatically use this backend unless the backend configuration changes. Initializing provider plugins... - Finding latest version of hashicorp/oci... - Installing hashicorp/oci v6.36.0... - Installed hashicorp/oci v6.36.0 (signed by HashiCorp) Terraform has been successfully initialized!
You can now write Terraform plan/create/destroy commands and you will find that your terraform.tfstate file is mastered inside your object storage bucket.
While an operation is running, you will also see a lock file present in the same bucket and the existence of that lock file will prevent concurrent operations. Once an operation completes or is terminated, the lock file is removed from the bucket.
If another user attempts to perform a Terraform operation while your operation is running, they will receive the following error and need to back-off and retry.
$> terraform apply ╷ │ Error: Error acquiring the state lock │ │ Error message: Error returned by ObjectStorage Service. Http Status Code: 412. Error Code: IfNoneMatchFailed. Opc │ request id: iad-1:pi_B30zba7upYMZiKZw-fz3Zhj6Y5ZcxSX3YoPqKlzWW0Pr66wnuJFzsyc6MAqQC. | Message: The If-None-Match header is '*' but there is an existing entity │ Operation Name: PutObject │ Timestamp: 2025-04-25 15:44:35 +0000 GMT │ Client Version: Oracle-GoSDK/65.89.1 │ Request Endpoint: PUT
If you want to implement token-based authentication, you will need to modify your backend block and provider blocks to look something like this:
provider "oci" {
auth = "SecurityToken"
config_file_profile = "tf"
region = "us-ashburn-1"
}
terraform {
backend "oci" {
bucket = "tf-remote-state"
namespace = "foobar"
auth = "SecurityToken"
config_file_profile = "tf"
region = "us-ashburn-1"
}
}
While the bucket and namespace attributes don’t change, notice that three new attributes are required in both the provider and backend blocks:
- auth: This must be SecurityToken
- config_file_profile: A key for your OCI config file (~/.oci/config) corresponding to your SecurityToken profile. Choose a unique name here that will be used when you authenticate via the CLI.
- region: The designator for the OCI region whose APIs you’re calling and in which your object storage bucket has been created
Before interacting with Terraform, you need to first authenticate with OCI and receive a short-lived token. Issuing the oci session authenticate command with the region and config_file_profile matching your Terraform configuration will redirect you to a browser, allow you to authenticate, and return you to your shell with a short-lived token placed inside you session configuration.
$> oci session authenticate --region us-ashburn-1 --session-expiration-in-minutes 60 --profile-name tf
Please switch to newly opened browser window to log in! You can also open the following URL in a web browser window to continue:
https://login.us-ashburn-1.oraclecloud.com/v1/oauth2/authorize?action=login&client_id=iaas_console&response_type=token+id_token&nonce=[redacted]&scope=openid&public_key=[redacted]]&redirect_uri=http%3A%2F%2Flocalhost%3A8181
Completed browser authentication process!
Config written to: /Users/eshneken/.oci/config
Try out your newly created session credentials with the following example command:
oci iam region list --config-file /Users/eshneken/.oci/config --profile tf --auth security_token
You may now run Terraform plan/apply/destroy operations without the need to position long lived API keys on your local computer.
Conclusion
If you’ve been waiting for a multi-cloud friendly way to manage your remote Terraform state in a concurrency & security conscious fashion, the wait is over. Get the latest versions of Terraform, review the backend documentation, and go native today!