Fails to upload state when using shared credentials file

[ljfranklin]

Fails to upload state when using shared credentials file

[ljfranklin]

@minniux my guess is the backend_config.shared_credentials_file path might be incorrect. The path should be relative to the terraform_source dir I think, so you'd need something like ``backend_config.shared_credentials_file: ../creds/creds.json. If that doesn't work posting your pipeline config's source` and `put` sections would help debug further.

[minniux]

@ljfranklin thank you for the fast reply, I've tried setting the credentials file relative to terraform_source as advised, but still got the same error.

Here are the source and put sections I'm using:

resources:
  - name: example-stack
    type: terraform
    source:
      backend_type: s3
      backend_config:
        bucket: example-bucket
        key: example/main
        region: eu-west-1
        profile: ((AWS_PROFILE_NAME))
        shared_credentials_file: ((CREDENTIALS_FILE))

  - put: example-stack
    params:
      env_name: staging
      terraform_source: terraform/example-stack
      env:
        AWS_SHARED_CREDENTIALS_FILE: ((CREDENTIALS_FILE))
        AWS_DEFAULT_REGION: eu-west-1
        AWS_PROFILE: ((AWS_PROFILE_NAME))

Note that If put.env is removed it fails to read the state, and if it's replaced with source.env it gives the same behavior to read the state and apply changes without uploading state

[ljfranklin]

As far as I can tell from reading the docs here and the terraform source code, it looks like shared_credentials_file must be a path to a file on disk rather than the contents of the file. Since the Concourse design doesn't let you pass files to check tasks, the resource's check calls will always fail. To use profiles and the shared credential file I think you'd have to open a PR or feature request to Terraform.

That said, using access_key and secret_key should work as expected.

[minniux]

shared_credentials_file is referring to a file, but the file gets written in a task before the put task (that's to generate the session token), so that's why the check is failing
Although I'm not totally sure, is the failure in saving state in apply related to check failure?

But anyway I'll look for another authentication method to use for our case, thanks for helping :)

[ljfranklin]

Happy to help! Going to close this out as there's unfortunately not anything I can change in the resource to support the shared_credentials_file use case. In the past I've submitted PRs to terraform providers to take either a filepath or file contents for params like shared_credentials_file, so if you submit a PR to Terraform they'd probably accept it.