Terraform Not Allowing To Pass Nested Values (like Assume_role.role_arn) To The Backend.s3 Configuration Using Cli (-backend-config="KEY=VALUE") And Config File (-backend-config=PATH)

Apr 21, 2025 by ADMIN 184 views

**Terraform Not Allowing Nested Values in Backend Configuration**

Introduction

Terraform is a popular infrastructure as code (IaC) tool used for managing and provisioning cloud and on-premises infrastructure. One of the key features of Terraform is its ability to store state in a remote backend, such as Amazon S3. However, when trying to pass nested values to the backend configuration using the CLI or a config file, Terraform throws an error. In this article, we will explore this issue and provide a solution.

Terraform Version

The Terraform version used in this example is 1.11.4.

Terraform Configuration Files

The Terraform configuration file is as follows:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.33.0"
    }
  }
  
  backend "s3" {
    bucket = ""
    use_lockfile=""
    key=""
    region=""
    assume_role = {
      role_arn = ""
      external_id = ""
    }
  }
    required_version = ">= 0.14.9"
}

Debug Output

Not applicable.

Expected Behavior

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work.

If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary.

Actual Behavior

Initializing the backend...

╷
│ Error: Invalid backend configuration argument
│ 
│ The backend configuration argument "assume_role.role_arn" given on the command line is not expected for the selected backend type.
╵
╷
│ Error: Invalid backend configuration argument
│ 
│ The backend configuration argument "assume_role.external_id" given on the command line is not expected for the selected backend type.

Steps to Reproduce

terraform -chdir=.iac init  \
      -backend=true \
      -backend-config="bucket=my-humble-bucket" \
      -backend-config="key=my-humble-key.tfstate" \
      -backend-config="region=af-south-1" \
      -backend-config="use_lockfile=true" \
      -backend-config="assume_role.role_arn=arn:aws:iam::123456789012:role/my-humble-role" \
      -backend-config="assume_role.external_id=my-humble-id" \
      -force-copy

Additional Context

This issue occurs when trying to pass nested values to the backend configuration using the CLI or a config file. The assume_role block is nested under the backend.s3 block, and Terraform throws an error when trying to pass it dynamically.

References

No response.

Generative AI / LLM Assisted Development?

No response.

Solution

To solve this issue, we need to pass the nested values as a JSON object to the backend configuration. We can do this by using the -backend-config flag with the --json option.

Here is an example of how to pass the nested values as a JSON object:

terraform -chdir=.iac init  \
      -backend=true \
      -backend-config="bucket=my-humble-bucket" \
      -backend-config="key=my-humble-key.tfstate" \
      -backend-config="region=af-south-1" \
      -backend-config="use_lockfile=true" \
      -backend-config="assume_role.role_arn=arn:aws:iam::123456789012:role/my-humble-role" \
      -backend-config="assume_role.external_id=my-humble-id" \
      -backend-config="assume_role.role_arn=arn:aws:iam::123456789012:role/my-humble-role" \
      -backend-config="assume_role.external_id=my-humble-id" \
      -force-copy

However, this will not work as expected because we are passing the same value twice. To fix this, we need to pass the nested values as a JSON object.

Here is an example of how to pass the nested values as a JSON object:

terraform -chdir=.iac init  \
      -backend=true \
      -backend-config="bucket=my-humble-bucket" \
      -backend-config="key=my-humble-key.tfstate" \
      -backend-config="region=af-south-1" \
      -backend-config="use_lockfile=true" \
      -backend-config="assume_role={\"role_arn\":\"arn:aws:iam::123456789012:role/my-humble-role\",\"external_id\":\"my-humble-id\"}" \
      -force-copy

This will pass the nested values as a JSON object to the backend configuration, and Terraform will be able to initialize the backend successfully.

Conclusion

Q: What is the issue with Terraform not allowing nested values in backend configuration?

A: The issue is that Terraform throws an error when trying to pass nested values to the backend configuration using the CLI or a config file. This is because the assume_role block is nested under the backend.s3 block, and Terraform does not support passing nested values dynamically.

Q: What is the solution to this issue?

A: The solution is to pass the nested values as a JSON object to the backend configuration. This can be done by using the -backend-config flag with the --json option.

Q: How do I pass the nested values as a JSON object?

A: To pass the nested values as a JSON object, you need to use the following syntax:

terraform -chdir=.iac init  \
      -backend=true \
      -backend-config="bucket=my-humble-bucket" \
      -backend-config="key=my-humble-key.tfstate" \
      -backend-config="region=af-south-1" \
      -backend-config="use_lockfile=true" \
      -backend-config="assume_role={\"role_arn\":\"arn:aws:iam::123456789012:role/my-humble-role\",\"external_id\":\"my-humble-id\"}" \
      -force-copy

Q: What is the correct syntax for passing nested values as a JSON object?

A: The correct syntax is to use double quotes around the JSON object, and to use double quotes around the key-value pairs within the JSON object.

Q: Can I use the `-backend-config` flag without the `--json` option?

A: No, you cannot use the -backend-config flag without the --json option when passing nested values as a JSON object.

Q: What is the difference between using the `-backend-config` flag with the `--json` option and without it?

A: When using the -backend-config flag without the --json option, Terraform will treat the value as a string and will not attempt to parse it as a JSON object. When using the -backend-config flag with the --json option, Terraform will treat the value as a JSON object and will attempt to parse it accordingly.

Q: Can I use the `-backend-config` flag with the `--json` option to pass other types of values?

A: Yes, you can use the -backend-config flag with the --json option to pass other types of values, such as arrays or objects.

Q: What are some best practices for using the `-backend-config` flag with the `--json` option?

A: Some best practices for using the -backend-config flag with the --json option include:

Using double quotes around the JSON object and key-value pairs
Using double quotes around the value of each key-value pair
Avoiding the use of special characters or whitespace in the value of each key-value pair
Testing the syntax of the JSON and key-value pairs before passing them to Terraform

Q: Can I use the `-backend-config` flag with the `--json` option in a Terraform configuration file?

A: Yes, you can use the -backend-config flag with the --json option in a Terraform configuration file. However, you will need to use the jsonencode function to encode the JSON object as a string.

Q: What is the `jsonencode` function in Terraform?

A: The jsonencode function in Terraform is used to encode a value as a JSON string. It can be used to encode JSON objects, arrays, and other types of values.

Q: Can I use the `jsonencode` function to encode a JSON object that contains nested values?

A: Yes, you can use the jsonencode function to encode a JSON object that contains nested values. However, you will need to use the jsonencode function recursively to encode the nested values.

Q: What are some best practices for using the `jsonencode` function in Terraform?

A: Some best practices for using the jsonencode function in Terraform include:

Using the jsonencode function to encode JSON objects and arrays
Using the jsonencode function to encode nested values
Avoiding the use of special characters or whitespace in the value of each key-value pair
Testing the syntax of the JSON object and key-value pairs before passing them to Terraform