archive_file changes file permissions

[hashicorp]

archive_file changes file permissions

[paultyng]

We talked a bit more about it internally, and we are thinking this all potentially may be better on the AWS lambda resource itself, so that instead of S3 or a local zip, you can specify the files right there and it will create a zip with all the necessary settings for Lambda. If you would prefer that functionality, please open that issue on the AWS provider.

[yermulnik]

Inadvertently this broke recently deployed Python scripts for me:

START RequestId: b477b5d9-2dd8-11e8-abd7-adbc37f1bf90 Version: $LATEST
module initialization error: [Errno 13] Permission denied: '/var/task/lambda_function.py'

END RequestId: b477b5d9-2dd8-11e8-abd7-adbc37f1bf90

Lambda seems to require world-readable permissions and previous behavior of archive_file was appropriate/suitable for at least Python scripts (presumably for any non-binary: nodejs, python, etc).
So to workaround this at the moment I'm going to use null_resource with local-exec provisioner to chmod a+r lambda_function.py before archiving (to ensure file has appropriate perms).
Might be a good idea to add an optional parameter for archive_file to allow people to set specific permissions on files before adding them to archive.

[OliverEhrhardt]

Fixed this issue by using source_dir and placing the files I wanted to archive in their own directory, instead of the source blocks I used above. Looks like referencing content within the source block (and I would assume source_content in the base archive_file block as well, but I haven't tested that) creates an entirely new file with the content provided and adds that file to the archive. I'm not sure if that's what is happening under the hood, but to me it seems that way based on the behavior. I feel like something should be mentioned in the documentation to clear things up for people if this is intended.

[roberterdin]

Something along the lines of the following works as a workaround...

data "external" "compile_and_zip_lambda" {
  program = ["bash", "${path.module}/build_for_aws.sh", "${path.module}"]
}

build_for_aws.sh:

#!/usr/bin/env bash
set -e

if [[ "$1" != "" ]]; then
    DIR="$1"
else
    DIR=.
fi

# make sure you have the `-q` flag to not mess with the output JSON
zip -jq ${DIR}/your_zip ${DIR}/your_input_dir
BASE_64_SHA256=$(shasum -a 256 -p ${DIR}/your_zip | base64)
echo "{ \"source_hash\": \"${BASE_64_SHA256}\"}"

[OliverEhrhardt]

This issue is still occurring on Mac OS X Catalina with Terraform 0.12.19, using archive_file to zip multiple files like this

data "archive_file" "docs_archive" {
  type        = "zip"
  output_path = "${path.module}/function.zip"

  source {
    content = data.local_file.bootstrap.content
    filename = "bootstrap"
  }

  source {
    content = data.local_file.function.content
    filename = "function.sh"
  }
}

The original files had -rwxr-xr-x permissions but when I unzip and check the files those permissions get reset to -rw-r--r--.

This makes custom runtimes in AWS Lambda not work due to permission errors. Is there anyway to reference a zip file without archive_file since the permissions are preserved when just using zip?

[kmoe]

The workaround in #90 has been released in terraform-provider-archive v2.2.0. If output_file_mode does not solve your problem, please comment on this issue or open a new one.

[paultyng]

The fix to this is included in v1.0.1 and was released earlier today.

[yermulnik]

@paultyng should I raise a new issue for this (optional parameter to set perms before archiving)?

[KyleKotowick]

This issue is still occurring on Windows. Within the ZIP file created by archive_file when running Terraform v0.12.18 with AWS provider v2.43 on Windows 10, the contained file has 666 permissions (no execute). Running the exact same Terraform plan with the same version on Linux results in the contained file having 777 permissions.

[nick-alloy]

Same thing here on Linux. Executable bits are being unset.

$ terraform --version
Terraform v0.12.23
+ provider.archive v1.3.0

[nick-alloy]

Thanks, @OliverEhrhardt. That worked for me, as well.

[jowrjowr]

this example definitely suffers from this issue:

data "archive_file" "modify_dms_instance" {
  type        = "zip"
  output_path = "${path.module}/lambda/modify_dms_instance.zip"

  source {
    content  = file("${path.module}/lambda/bootstrap")
    filename = "bootstrap"
  }

  source {
    content  = file("${path.module}/lambda/modify_dms_instance.sh")
    filename = "main.sh"
  }
}

this, however, worked fine:

data "archive_file" "modify_dms_instance" {
  type        = "zip"
  output_path = "${path.module}/lambda/modify_dms_instance.zip"
  source_dir  = "${path.module}/lambda/modify_dms_instance/"
}

[dinvlad]

Weirdly, I see the file permissions to be preserved, but I would like them to become 644 (so that the deployment is completely reproducible). Would it be possible to add an optional flag to set file permissions via archive_file?

[mancej]

Yup, this is not happening on my Mac, but it is happening on our CentOS build server and it it's driving me bonkers.

[artis3n]

I had no issue with file permissions when I used source { content ... } but moving to source_dir, I am seeing a binary being invoked by my lambda function's permissions changing from 0755 to 0666.

[alan-w-fanduel]

output_file_mode did not fix it for me.

The tf worked in development but was running across permission denied issue in the CI/CD. Creating a zip file in the /tmp folder worked for me as the tmp folder has permissions of 777 and the created zip will also have 777 permissions.

locals {
  zip_file = "/tmp/some_zip.zip"
}

data "archive_file" "lambda_zip" {
  type        = "zip"
  source_file = "${path.module}/lambda.py"
  output_path = local.zip_file
}

resource "aws_lambda_function" "function" {
  function_name    = "${var.function_name}"
  filename         = "${data.archive_file.lambda_zip.output_path}"
  source_code_hash = "${data.archive_file.lambda_zip.output_base64sha256}"

[eugbyte]

output_file_mode works for me on windows when i set it to 0777, that is, grant permission to make the file executable.

// build the binary for the lambda function in a specified path
resource "null_resource" "fn_subscription_binary" {
  provisioner "local-exec" {
    command = "env GOOS=linux go build -o ${local.binary_path_subscription_fn} -ldflags='-s -w' ${local.src_path_subscription_fn}"
  }
}

// zip the binary, as we can use only zip files to AWS lambda
data "archive_file" "fn_subscription_archive" {
  depends_on = [null_resource.chmod]

  type             = "zip"
  source_file      = local.binary_path_subscription_fn
  output_path      = local.archive_path_subscription_fn
  output_file_mode = "0777"   // grant permission to make file executable for linux environment
}

When I experimented with other chmod values that do not grant execution, e.g. 0666, I get the error message {"errorMessage":"fork/exec /var/task/main: permission denied","errorType":"PathError"}