aws/aws-sam-build-images

AWS SAM build images

AWS SAM build images

AWS SAM build container images for all supported runtimes to build and deploy serverless applications on AWS.

Security

See CONTRIBUTING for more information.

License

This project is licensed under the Apache-2.0 License.

aws-sam-build-images's Issues

Update Go?

Golang's current stable version is 1.17.5, the current image is still on 1.16.5. Can we get this version updated?

Add YUM to all build images

Bug: cannot use pycrypto with python3.11

Description:

When trying to build/test an 3.11 Python application that requires pycrypto, there is this issue:

[...]
    gcc -pthread -Wsign-compare -fwrapv -Wall -fPIC -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/usr/include/ -I/var/lang/include/python3.11 -c src/_fastmath.c -o build/temp.linux-x
86_64-cpython-311/src/_fastmath.o
      In file included from /var/lang/include/python3.11/Python.h:12:0,
                       from src/_fastmath.c:31:
      /var/lang/include/python3.11/pyconfig.h:1784:0: warning: "_POSIX_C_SOURCE" redefined
       #define _POSIX_C_SOURCE 200809L
     
      In file included from /usr/include/bits/libc-header-start.h:33:0,
                       from /usr/include/stdio.h:27,
                       from src/_fastmath.c:29:
      /usr/include/features.h:289:0: note: this is the location of the previous definition
       # define _POSIX_C_SOURCE 199506L
     
      src/_fastmath.c:33:10: fatal error: longintrepr.h: No such file or directory
       #include <longintrepr.h>    /* for conversions */
                ^~~~~~~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

The issue doesn't occur with Python3.9 image.

Steps to reproduce:

1. Create an application with SAM using Python3.11 runtime
2. add pycrypto in the requirements.txt file
3. Build the application

Observed result:

      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

Expected result:

The application build correctly.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: Docker
2. If using SAM CLI, sam --version: 23.1.2
3. AWS region: /

Feature request: Add docker to images

Describe your idea/feature/enhancement

I wish AWS SAM build images would have Docker installed on them.

For Lambdas that are run with a container (i.e. PackageType=Image), AWS SAM can build and push images to ECR (or other image repos). But you can only do this if docker is installed, and if you are logged in with docker. In order for the AWS SAM build images to support container-lambdas being built and pushed with AWS SAM, it would be great to have docker installed on them.

I am trying to sam package an image to ECR from a CI/CD CodePipeline, similar to the process described
here.

I'm using sam/build-python3.9, where docker is not installed.

Obvious workarounds are either to create your own image off these, or to install and start docker when running the image. But given that pushing images is an AWS SAM feature, it would be great for these images to support that feature out of the box.

Missing Yum on Node Build Images

I have noticed that both of these images are missing yum where all the other SAM build images have it.

• amazon/aws-sam-cli-build-image-nodejs12.x
• amazon/aws-sam-cli-build-image-nodejs14.x

Yum is also installed on each Lambda runtime public.ecr.aws/lambda/... (including the node versions above) images and our tooling requires that they also be on the build image. Thinking it is a big oversight that somehow these are missing?

Bug: build-nodejs20.x requires LD_LIBRARY_PATH= to run SAM CLI commands

Description:

The build-nodejs20.x variant of the image breaks when running sam commands such as sam validate unless the environment variable LD_LIBRARY_PATH is set and empty.

This is mentioned here:

aws-sam-build-images/tests/build_image_base_test.py

Line 92 in 1ef7557

# For nodejs20.x set LD_LIBRARY_PATH env variable to execute sam commands

Is it expected that users of the nodejs20.x image will need to set LD_LIBRARY_PATH= in order to run sam commands, and if so is this documented somewhere? Is this a temporary situation?

Steps to reproduce:

1. Run the following command:

   docker run -it --rm -w /work -v $(pwd):/work public.ecr.aws/sam/build-nodejs20.x:1.103 sam validate --lint
   

Observed result:

2023-11-18 05:55:33 Command 'validate' is not configured correctly. Unable to import 'samcli.commands.validate.validate'
Traceback (most recent call last):
  File "/usr/local/opt/sam-cli/lib64/python3.9/site-packages/samcli/cli/command.py", line 263, in get_command
    mod = importlib.import_module(pkg_name)
  File "/usr/lib64/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/usr/local/opt/sam-cli/lib64/python3.9/site-packages/samcli/commands/validate/validate.py", line 6, in <module>
    import boto3
  File "/usr/local/opt/sam-cli/lib64/python3.9/site-packages/boto3/__init__.py", line 17, in <module>
    from boto3.session import Session
  File "/usr/local/opt/sam-cli/lib64/python3.9/site-packages/boto3/session.py", line 17, in <module>
    import botocore.session
  File "/usr/local/opt/sam-cli/lib64/python3.9/site-packages/botocore/session.py", line 26, in <module>
    import botocore.client
  File "/usr/local/opt/sam-cli/lib64/python3.9/site-packages/botocore/client.py", line 15, in <module>
    from botocore import waiter, xform_name
  File "/usr/local/opt/sam-cli/lib64/python3.9/site-packages/botocore/waiter.py", line 18, in <module>
    from botocore.docs.docstring import WaiterDocstring
  File "/usr/local/opt/sam-cli/lib64/python3.9/site-packages/botocore/docs/__init__.py", line 15, in <module>
    from botocore.docs.service import ServiceDocumenter
  File "/usr/local/opt/sam-cli/lib64/python3.9/site-packages/botocore/docs/service.py", line 14, in <module>
    from botocore.docs.client import (
  File "/usr/local/opt/sam-cli/lib64/python3.9/site-packages/botocore/docs/client.py", line 18, in <module>
    from botocore.docs.example import ResponseExampleDocumenter
  File "/usr/local/opt/sam-cli/lib64/python3.9/site-packages/botocore/docs/example.py", line 13, in <module>
    from botocore.docs.shape import ShapeDocumenter
  File "/usr/local/opt/sam-cli/lib64/python3.9/site-packages/botocore/docs/shape.py", line 19, in <module>
    from botocore.utils import is_json_value_header
  File "/usr/local/opt/sam-cli/lib64/python3.9/site-packages/botocore/utils.py", line 38, in <module>
    import botocore.httpsession
  File "/usr/local/opt/sam-cli/lib64/python3.9/site-packages/botocore/httpsession.py", line 22, in <module>
    from urllib3.util.ssl_ import (
ImportError: cannot import name 'ssl' from 'urllib3.util.ssl_' (/usr/local/opt/sam-cli/lib64/python3.9/site-packages/urllib3/util/ssl_.py)
Usage: sam [OPTIONS] COMMAND [ARGS]...
Try 'sam -h' for help.

Error: No such command 'validate'.

Expected result:

Running the following command:

docker run -it --rm -e 'LD_LIBRARY_PATH=' -w /work -v $(pwd):/work public.ecr.aws/sam/build-nodejs20.x:1.103 sam validate --lint

instead produces the following expected output (modulo the RuntimeWarning):

        SAM CLI now collects telemetry to better understand customer needs.

        You can OPT OUT and disable telemetry collection by setting the
        environment variable SAM_CLI_TELEMETRY=0 in your shell.
        Thanks for your help!

        Learn More: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-telemetry.html

/usr/local/opt/sam-cli/lib64/python3.9/site-packages/networkx/utils/backends.py:135: RuntimeWarning: networkx backend defined more than once: nx-loopback
  backends.update(_get_backends("networkx.backends"))
/work/template.yaml is a valid SAM Template

[Question] enable unit test on Bitbucket Pipelines and using npm/npx

Hi there!

I'm using sam pipeline init --bootstrap and have two bellow concerns, could you please help me to clarify:
1, How to run npm in pipeline?
I need to implement migrate database step on Bitbucket Pipelines. I'm using sequelize-cli so I need to run a command something like npx sequelize-cli db:migrate
But it seems node is not existing in this image. So I tried to install node 18 by myself but it throws this error GLIBC_2.28' not found.
I downgrade to node 16 and it is ok now. But this is not what I expected.
So could you please tell me what is the best way to use npm command in this case?

2, How can I enable unit test on Bitbucket Pipelines?
I saw the document how to enable unit test here but it's just for CodePipeline and GitHub. So how about Bitbucket Pipelines please?

Many thanks!

Bug: UseNpmCi broken in build-nodejs18.x:1.85-x86_64

Description:

Using UseNpmCi in build-nodejs18.x:1.85-x86_64

Metadata:
    BuildMethod: esbuild
    BuildProperties:
        UseNpmCi: True

causes this error

$ sam build --template template.yaml --use-container

Build Failed
Error: NodejsNpmEsbuildBuilder:EsbuildBundle - Esbuild Failed: ✘ [ERROR] Invalid build flag: "--use-npm-ci"

When using an older version it works. public.ecr.aws/sam/build-nodejs18.x:1.84-x86_64

$ sam build --template template.yaml --use-container --build-image public.ecr.aws/sam/build-nodejs18.x:1.84-x86_64

Steps to reproduce:

Use UseNpmCi and build using public.ecr.aws/sam/build-nodejs18.x:1.85.0-x86_64 which is latest as of writing

Observed result:

Fetching public.ecr.aws/sam/build-nodejs18.x:latest-x86_64 Docker container image......

...OMITTED...

NodejsNpmEsbuildBuilder:CopySource succeeded
Running NodejsNpmEsbuildBuilder:NpmInstall
NODEJS installing in: /tmp/samcli/scratch
executing NPM: ['npm', 'install', '-q', '--no-audit', '--no-save', '--unsafe-perm', '--production']
NodejsNpmEsbuildBuilder:NpmInstall succeeded
Running NodejsNpmEsbuildBuilder:EsbuildBundle
NODEJS building ['/tmp/samcli/scratch/handler.ts'] using esbuild to /tmp/samcli/artifacts
Using the following default args: ['--bundle', '--platform=node', '--outdir=/tmp/samcli/artifacts', '--format=cjs']
Configuring the parameter 'use_npm_ci=True'
Configuring the parameter 'minify=True'
Configuring the parameter 'target=es2020'
Configuring the parameter 'sourcemap=True'
Found the following args in the config: ['--use-npm-ci', '--minify', '--target=es2020', '--sourcemap']
checking for esbuild in: ['/tmp/samcli/scratch/node_modules/.bin']
potential esbuild binaries: ['/tmp/samcli/scratch/node_modules/.bin/esbuild']
executing Esbuild: ['/tmp/samcli/scratch/node_modules/.bin/esbuild', 'handler.ts', '--bundle', '--platform=node', '--outdir=/tmp/samcli/artifacts', '--format=cjs', '--use-npm-ci', '--minify', '--target=es2020', '--sourcemap']
NodejsNpmEsbuildBuilder:EsbuildBundle failed
Traceback (most recent call last):
  File "/usr/local/opt/sam-cli/lib64/python3.7/site-packages/aws_lambda_builders/workflows/nodejs_npm_esbuild/actions.py", line 98, in execute
    self._subprocess_esbuild.run(args, cwd=self._working_directory)
  File "/usr/local/opt/sam-cli/lib64/python3.7/site-packages/aws_lambda_builders/workflows/nodejs_npm_esbuild/esbuild.py", line 97, in run
    raise EsbuildExecutionError(message=err.decode("utf8").strip())
aws_lambda_builders.workflows.nodejs_npm_esbuild.exceptions.EsbuildExecutionError: Esbuild Failed: ✘ [ERROR] Invalid build flag: "--use-npm-ci"

Expected result:

Expected it to build.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: Windows 11
2. If using SAM CLI, sam --version: SAM CLI, version 1.84.0
3. AWS region: not relevant

Yarn

Any chance we cannot get yarn added into these containers as well?

What happened with aws-sam-cli-build-image-nodejs16.x

Description:

Some month ago we was deploying our function using nodejs16 image version. Now it's not available anymore. The repository still contains the Dockerfile for building this image, but it looks like it's excluded to be pushed.

Add Go runtime

Add build-go1.x.

Make emulation base images available

CDK users have reported an issue with the sam build images that causes building to fail if they are using btrfs or zfs storage drivers. The suggested resolution is to update the root mode which can only be done in the base image.

Are those base image Dockerfiles (i.e. public.ecr.aws/sam/emulation-python3.9) available anywhere?

Access denied for image pull using latest SAM

Invoking com.somecompany.aws.handler.GatewayHandler (java8)
Image was not found.
Removing rapid images for repo public.ecr.aws/sam/emulation-java8
Building image......
Failed to build Docker Image
NoneType: None
Error: Error building docker image: Get "https://public.ecr.aws/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

This is using latest intellij with latest aws toolbox, with latest sam

Add .NET runtimes

Description

Currently there are not provided docker images for the .NET runtime, and the sam build command does not support the --use-container flag. This makes typical build and deploy workflows with SAM not working consistently for .NET projects. It would be great if a Docker image could be provided and the sam cli updated to work with the container.

Benefits

Ensures consistent behavior for all AWS Lambda supported runtimes by the AWS SAM cli and build/deploy workflow.

Request

Add .NET Dockerfiles supporting AWS runtimes

• Dockerfile-dotnetcore31
• Dockerfile-dotnetcore21

Add node16.x image to docker hub

Hello there!

I'd like to request a docker hub image for node 16.x
I wanted to make a pull request, as amazon is supporting node 16.x for lambdas from May https://aws.amazon.com/ru/about-aws/whats-new/2022/05/aws-lambda-adds-support-node-js-16/ but I can see a dockerfire, for node 16.x already in develop branch.

So, would be great if you can publish those..

Thanks in advance!

docker image does not include libffi.so.6 needed for Python 3.11

Description:

libffi.so.6 is missing from the python3.11 latest image

using vscode launch config to run sam local invoke create the image and acquires public.ecr.aws/sam/emulation-python3.11. after mounting the image and starting the code, if fails to start the session and the output presents
Traceback (most recent call last): File "/tmp/lambci_debug_files/py_debug_wrapper.py", line 46, in <module> runpy.run_module('debugpy', run_name='__main__') File "<frozen runpy>", line 229, in run_module File "<frozen runpy>", line 88, in _run_code File "/var/task/debugpy/__main__.py", line 37, in <module> from debugpy.server import cli File "/var/task/debugpy/server/__init__.py", line 7, in <module> import debugpy._vendored.force_pydevd # noqa ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/task/debugpy/_vendored/force_pydevd.py", line 44, in <module> preimport('pydevd', [ File "/var/task/debugpy/_vendored/__init__.py", line 126, in preimport import_module(name) File "/var/lang/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/task/debugpy/_vendored/pydevd/pydevd.py", line 45, in <module> from _pydevd_bundle import pydevd_utils File "/var/task/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_utils.py", line 9, in <module> import ctypes File "/var/lang/lib/python3.11/ctypes/__init__.py", line 8, in <module> from _ctypes import Union, Structure, Array ImportError: libffi.so.6: cannot open shared object file: No such file or directory 05 Dec 2023 21:07:19,654 [ERROR] (rapid) Init failed error=Runtime exited with error: exit status 1 InvokeID=

Steps to reproduce:

attached is a zip file of the sample lambda created with sam init (Note: I included 3.11, and 3.9 venv for reference)
lambda-test.zip
actions to produce the error
1- in the template .yaml; change the function Runtime: from python3.9 to python3.11
2- from vscode debug, launch lambda-test:HelloWorldFunction (python3.11)
Note: the error disappears when launch vscode after changing the Runtime to python3.9 in template.yaml

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: MacOS
2. If using SAM CLI, sam --version: 1.99.0
3. AWS region: us-east-1

Please add rsync to images

Hey folks, hoping someone can step in and add rsync to the list of packages that yum installs. Tried to get around this using yum in the commands I'm passing to the container, but there's some funkiness going on there and bash can't find it.

Running SAM CLI commands within the Nodejs20.x Build Image

NOTE: This issue only impacts use cases that use the Nodejs20.x Build Image to run SAM CLI commands excluding sam build. If you are using other Build Images or are using sam build --use-container for your Nodejs20.x Lambda applications, you are not impacted.

Due to different OpenSSL versions required by Node.js and Python within the Nodejs20.x Build Image, SAM CLI commands that involve making network calls (e.g. sam init, sam deploy) will fail. We are working to fix the issue.

For workaround, if your use case requires using these impacted commands within the Nodejs20.x Build Image, please set the environment variable LD_LIBRARY_PATH to empty for that command. For example, LD_LIBRARY_PATH= sam deploy.

Dockerfile-java11 - Bump maven to 3.9.5

Describe your idea/feature/enhancement

Bump maven version to 3.9.5 for java11 image.
I am running into issues using Maven 3.8.8, would appreciate if the version could be bumped to 3.9.5
Some dependencies in my Java project(Quarkus) cause issues when trying to retrieve dependencies from corporate mirror of maven central repository. I get 403 Forbidden responses from the server. Maven 3.8.8 uses a class called WagonTransporter which has a conflict with a version of WagonTransporter included in Quarkus. Maven 3.9+ moved to a different class called HttpTransporter. This seemed to fix the problem for me.

Link to issue:
quarkusio/quarkus#33220

I recognized you might have reasons to keep Maven at its current version, any chance of having 2 different java11 images?
Thanks for your consideration.

Proposal

In Dockerfile-java11
line 62: replace maven URL with: https://downloads.apache.org/maven/maven-3/3.9.5/binaries/apache-maven-3.9.5-bin.tar.gz
line 65: replace reference to apache-maven-3.8.8 with reference to apache-maven-3.9.5

Possible to update pip in the public.ecr.aws/sam/build-python3.6 build image?

The version of pip installed in the public.ecr.aws/sam/build-python3.6 image currently is 18.1. As mentioned in this psycopg2 project issue, older versions of pip (which include 18.1) fail to install the latest psycopg2 versions (2.9 or later).

If we upgrade to the latest pip version, 21.1.2, we are able to install the later psycopg2 versions without issues. Would it be possible to upgrade the default pip version in the public.ecr.aws/sam/build-python3.6 image to the latest version?