aws-samples/cookiecutter-aws-sam-python

A Cookiecutter template to create a Serverless App based on Serverless Application Model (SAM) and Python 3.8

Cookiecutter SAM for Python Lambda functions

Archived

This project is now archived. Please use the official AWS SAM CLI template instead.

sam init --app-template hello-world-powertools-python --name sam-app --package-type Zip --runtime python3.12 --no-tracing

---

This is a Cookiecutter template to create a Serverless App based on Serverless Application Model (SAM) and Python 3.9.

It is important to note that you should not try to git clone this project but use SAM CLI instead as {{cookiecutter.project_slug}} will be rendered based on your input and therefore all variables and files will be rendered properly.

Usage

Generate a new SAM based Serverless App: sam init --location gh:aws-samples/cookiecutter-aws-sam-python

You'll be prompted a few questions to help this cookiecutter template to scaffold this project and after its completed you should see a new folder at your current path with the name of the project you gave as input.

Options

Option	Description
include_safe_deployment	Sends by default 10% of traffic for every 1 minute to a newly deployed function using CodeDeploy + SAM integration - Linear10PercentEvery1Minute

Credits

• This project has been generated with Cookiecutter
• Bruno Alla's Lambda function template

License

This project is licensed under the terms of the MIT License with no attribution

cookiecutter-aws-sam-python's Issues

Template generation might fail if python environment is not setup correctly

Hi team,

A customer reported an issue while trying to generate a template with SAM CLI using this repository. You can read details here: aws/aws-sam-cli#5057

In short, their python environment was not setup correctly, therefore post hooks defined in this repository have failed to execute and template generation failed in the end.

I will recommend pointing this out specifically in the README file. We could also provide an FAQ kind of section where we can guide them correct setup to use this template while generating a new application.

Thanks!

Upgrade Requests library

Upgrade requests library in pipfile.lock to address CVE

Pipenv installing deps on system packages

As discussed with @jfuss pipenv from Makefile isn't entering the shell before it install all its dependencies, and that creates a dirty system packages which we were trying to be isolated from.

`make package` fails due to `ln` on macOS

On macOS 10.14.5 (18F132)

$ make package SERVICE=first_function
[*] Checking whether service first_function exists...
[*] Cleaning up local builds before build task...
[+] Packaging service 'first_function' -- This may take a while...
[*] Who needs all that anyway? Destroying environment....
rm -rf ./first_function/build
rm -rf ./first_function.zip
[+] Setting permissions for files under first_function/
[+] Setting permissions for directories under first_function/
[+] Cloning resource_records directory structure to first_function/build
[+] Cloning source files from resource_records to first_function/build
find first_function/ -type f -exec chmod go+r {} \;
find first_function/ -type d -exec chmod go+rx {} \;
rsync -a -f "+ */" -f "- *" first_function/ first_function/build/
rm -f .results.txt
[*] Checking whether service first_function exists...
[+] Installing 'first_function' dependencies..."
Requirement already satisfied: pipenv in /usr/local/lib/python3.7/site-packages (2018.11.26)
Requirement already satisfied: pip>=9.0.1 in /usr/local/lib/python3.7/site-packages (from pipenv) (19.0.3)
Requirement already satisfied: virtualenv in /usr/local/lib/python3.7/site-packages (from pipenv) (16.2.0)
Requirement already satisfied: setuptools>=36.2.1 in /usr/local/lib/python3.7/site-packages (from pipenv) (40.8.0)
Requirement already satisfied: virtualenv-clone>=0.2.5 in /usr/local/lib/python3.7/site-packages (from pipenv) (0.5.3)
Requirement already satisfied: certifi in /usr/local/lib/python3.7/site-packages (from pipenv) (2019.3.9)
Looking in indexes: https://pypi.python.org/simple
Collecting aws-xray-sdk==2.4.2 (from -r requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/58/f2/79f7918f4ddeec525742ddd4607abe4a82a29a6bc4c7e297995f59a18965/aws_xray_sdk-2.4.2-py2.py3-none-any.whl
Collecting boto3==1.9.158 (from -r requirements.txt (line 3))
  Using cached https://files.pythonhosted.org/packages/cd/b0/8e67d52f8787370828709fe4cad39184e65f3fec28023b78c2556a04be0c/boto3-1.9.158-py2.py3-none-any.whl
Collecting botocore==1.12.158 (from -r requirements.txt (line 4))
  Using cached https://files.pythonhosted.org/packages/18/21/1e6f12ef2642786bda935f9936bfe3a3f19466a1f3f7e079053421a39f0a/botocore-1.12.158-py2.py3-none-any.whl
Collecting docutils==0.14 (from -r requirements.txt (line 5))
  Using cached https://files.pythonhosted.org/packages/36/fa/08e9e6e0e3cbd1d362c3bbee8d01d0aedb2155c4ac112b19ef3cae8eed8d/docutils-0.14-py3-none-any.whl
Collecting future==0.17.1 (from -r requirements.txt (line 6))
Collecting jmespath==0.9.4 (from -r requirements.txt (line 7))
  Using cached https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl
Collecting jsonpickle==1.2 (from -r requirements.txt (line 8))
  Using cached https://files.pythonhosted.org/packages/07/07/c157520a3ebd166c8c24c6ae0ecae7c3968eb4653ff0e5af369bb82f004d/jsonpickle-1.2-py2.py3-none-any.whl
Collecting python-dateutil==2.8.0 (from -r requirements.txt (line 9))
  Using cached https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl
Collecting s3transfer==0.2.0 (from -r requirements.txt (line 10))
  Using cached https://files.pythonhosted.org/packages/d7/de/5737f602e22073ecbded7a0c590707085e154e32b68d86545dcc31004c02/s3transfer-0.2.0-py2.py3-none-any.whl
Collecting six==1.12.0 (from -r requirements.txt (line 11))
  Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting urllib3==1.25.3 (from -r requirements.txt (line 12))
  Using cached https://files.pythonhosted.org/packages/e6/60/247f23a7121ae632d62811ba7f273d0e58972d75e58a94d329d51550a47d/urllib3-1.25.3-py2.py3-none-any.whl
Collecting wrapt==1.11.1 (from -r requirements.txt (line 13))
Installing collected packages: jmespath, docutils, urllib3, six, python-dateutil, botocore, wrapt, jsonpickle, future, aws-xray-sdk, s3transfer, boto3
Successfully installed aws-xray-sdk-2.4.2 boto3-1.9.158 botocore-1.12.158 docutils-0.14 future-0.17.1 jmespath-0.9.4 jsonpickle-1.2 python-dateutil-2.8.0 s3transfer-0.2.0 six-1.12.0 urllib3-1.25.3 wrapt-1.11.1
ln: events: Is a directory
make[1]: *** [_install_deps] Error 1
make: *** [package] Error 2

$ ls -l events
total 8
-rw-r--r--  1 miketheman  staff  23 May 29 23:52 first_function_event.json

Stems from:

cookiecutter-aws-sam-python/{{ cookiecutter.project_name }}/Makefile

Line 149 in 2fa3f44

@ln -f events ${SERVICE}/build/events

macOS seems to disallow hard linking of directories, and the command fails.
See more at https://stackoverflow.com/questions/80875/what-is-the-unix-command-to-create-a-hardlink-to-a-directory-in-os-x

This makes commands like make build/package fail. Is there a decent workaround other than simply stopping that practice?

User's umask overrides permissions for sam initialized project

User's umask overrides permissions for sam initialized project

When a user's umask is to restrictive, projects initialized withsam init --location gh:aws-samples/cookiecutter-aws-sam-python results in projects that will not work when deployed to AWS.

Steps to reproduce

1. Download and install sam cli: pip install aws-sam-cli
2. Set your default permissions to a restrictive mode: umask 077
3. Initialize a project: sam init --location gh:aws-samples/cookiecutter-aws-sam-python
4. When prompted by the command in step 2. enter these details

       [+] Initializing project structure...
       project_name [Name of the project]: bar
       project_short_description [A short description of the project]:
       include_apigw [y]: n
       include_xray [y]:
       include_safe_deployment [y]:
       include_experimental_make [n]: y

5. Check the permissions on app.py

   $ ls -l first_function/app.py                                                                                                                                                                                                      
   -rw------- 1 johnt staff 1252 Jul  9 18:51 first_function/app.py
   $
   

6. Modify your umask to be less restrictive: umask 022
7. Repeat steps 3 & 4 this time calling your project bar2
8. Check the permissions on app.py

   $ ls -l first_function/app.py                                                                                                                                                                                                      
   -rw-r--r-- 1 johnt staff 1252 Jul  9 18:54 first_function/app.py
   $
   

The end result will be that the module or its resources will not load correctly because of the permissions.

unit tests failing due to xray not available locally

=========================================================== test session starts ===========================================================
platform darwin -- Python 3.6.5, pytest-3.5.1, py-1.5.3, pluggy-0.6.0 -- /Users/paxtod/.local/share/virtualenvs/sample-project--BvM5prf/bin/python
cachedir: .pytest_cache
rootdir: /Users/paxtod/WorkDocs/code/sample-project, inifile:
plugins: cov-2.5.1
collected 1 item

tests/unit/test_handler.py::test_lambda_handler FAILED [100%]

================================================================ FAILURES =================================================================
___________________________________________________________ test_lambda_handler ___________________________________________________________

apigw_event = {'body': '{ "test": "body"}', 'headers': {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,...ept-Language': 'en-US,en;q=0.8', 'Cache-Control': 'max-age=0', ...}, 'httpMethod': 'POST', 'path': '/examplepath', ...}

def test_lambda_handler(apigw_event):

  ret = app.lambda_handler(apigw_event, "")

tests/unit/test_handler.py:89:

---

first_function/app.py:27: in lambda_handler
message = get_message()
../../../.local/share/virtualenvs/sample-project--BvM5prf/lib/python3.6/site-packages/aws_xray_sdk/core/recorder.py:359: in wrapper
meta_processor=None,
../../../.local/share/virtualenvs/sample-project--BvM5prf/lib/python3.6/site-packages/aws_xray_sdk/core/recorder.py:367: in record_subsegment
subsegment = self.begin_subsegment(name, namespace)
../../../.local/share/virtualenvs/sample-project--BvM5prf/lib/python3.6/site-packages/aws_xray_sdk/core/recorder.py:231: in begin_subsegment
segment = self.current_segment()
../../../.local/share/virtualenvs/sample-project--BvM5prf/lib/python3.6/site-packages/aws_xray_sdk/core/recorder.py:215: in current_segment
entity = self.get_trace_entity()
../../../.local/share/virtualenvs/sample-project--BvM5prf/lib/python3.6/site-packages/aws_xray_sdk/core/recorder.py:316: in get_trace_entity
return self.context.get_trace_entity()
../../../.local/share/virtualenvs/sample-project--BvM5prf/lib/python3.6/site-packages/aws_xray_sdk/core/context.py:93: in get_trace_entity
return self.handle_context_missing()

---

self = <aws_xray_sdk.core.context.Context object at 0x103d352b0>

def handle_context_missing(self):
    """
        Called whenever there is no trace entity to access or mutate.
        """
    if self.context_missing == 'RUNTIME_ERROR':
        log.error(MISSING_SEGMENT_MSG)

      raise SegmentNotFoundException(MISSING_SEGMENT_MSG)

E aws_xray_sdk.core.exceptions.exceptions.SegmentNotFoundException: cannot find the current segment/subsegment, please make sure you have a segment open

../../../.local/share/virtualenvs/sample-project--BvM5prf/lib/python3.6/site-packages/aws_xray_sdk/core/context.py:118: SegmentNotFoundException
------------------------------------------------------------ Captured log call ------------------------------------------------------------
context.py 117 ERROR cannot find the current segment/subsegment, please make sure you have a segment open
======================================================== 1 failed in 1.07 seconds =========================================================

`sam init` fails to deploy `gh:aws-samples/cookiecutter-aws-sam-python` on Windows platform

Description:

A sam init fails to bootstrap a simple python hello-world template on Windows platform.
You may find this command example in sam init --help and also at https://github.com/aws-samples/cookiecutter-aws-sam-python:

sam init --location gh:aws-samples/cookiecutter-aws-sam-python

this command produces the following error:

'C:\Users\{USER}\AppData\Local\Temp\tmppht_go78' is not recognized as an internal or external command,
operable program or batch file.
2024-05-04 12:11:51 Stopping generation because post_gen_project hook script didn't exit successfully
Error: An error occurred while generating this project : Hook script failed (exit status: 1)

Steps to reproduce:

1. Run cmd (Win+R, cmd)
2. Run sam init --location gh:aws-samples/cookiecutter-aws-sam-python
3. Type in a project name: sam-demo

Observed result:

You've downloaded C:\Users\{USER}\.cookiecutters\cookiecutter-aws-sam-python before. Is it okay to delete and
re-download it? [y/n] (y): y
  [1/2] project_name (Name of the project): sam-demo
  [2/2] include_safe_deployment (n):
'C:\Users\{USER}\AppData\Local\Temp\tmppht_go78' is not recognized as an internal or external command,
operable program or batch file.
2024-05-04 12:11:51 Stopping generation because post_gen_project hook script didn't exit successfully
Error: An error occurred while generating this project : Hook script failed (exit status: 1)

Expected result:

A sample template should be deployed to sam-demo folder.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

Platform: Windows 11 Enterprise, 22H2, build 22621.3447

1. OS: Windows 11 Enterprise, 22H2, build 22621.3447
2. sam --version: SAM CLI, version 1.116.0
3. AWS region: eu-central-1

# Paste the output of `sam --info` here
{
  "version": "1.116.0",
  "system": {
    "python": "3.11.8",
    "os": "Windows-10-10.0.22621-SP0"
  },
  "additional_dependencies": {
    "docker_engine": "25.0.3",
    "aws_cdk": "Not available",
    "terraform": "Not available"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}

Add --debug flag to command you are running

D:\code>sam init --debug --location gh:aws-samples/cookiecutter-aws-sam-python
2024-05-04 12:32:51,916 | No config file found in this directory.
2024-05-04 12:32:51,922 | OSError occurred while reading TOML file: [Errno 2] No such file or directory: 'D:\\code\\samconfig.toml'
2024-05-04 12:32:51,926 | Config file location: D:\code\samconfig.toml
2024-05-04 12:32:51,928 | Config file 'D:\code\samconfig.toml' does not exist
2024-05-04 12:32:51,931 | OSError occurred while reading TOML file: [Errno 2] No such file or directory: 'D:\\code\\samconfig.toml'
2024-05-04 12:32:51,934 | Using config file: samconfig.toml, config environment: default
2024-05-04 12:32:51,937 | Expand command line arguments to:
2024-05-04 12:32:51,940 | --location=gh:aws-samples/cookiecutter-aws-sam-python --package_type=Zip --output_dir=.
2024-05-04 12:32:54,309 | Parameters dict created with input given
2024-05-04 12:32:54,311 | {'template': 'gh:aws-samples/cookiecutter-aws-sam-python', 'output_dir': '.', 'no_input': False,
'extra_context': {'architectures': {'value': ['x86_64']}}}
2024-05-04 12:32:54,314 | Baking a new template with cookiecutter with all parameters
You've downloaded C:\Users\{USER}\.cookiecutters\cookiecutter-aws-sam-python before. Is it okay to delete and re-download it? [y/n]
(y):
  [1/2] project_name (Name of the project): sam-demo
  [2/2] include_safe_deployment (n): ,
'C:\Users\{USER}\AppData\Local\Temp\tmppcr4666j' is not recognized as an internal or external command,
operable program or batch file.
2024-05-04 12:33:06 Stopping generation because post_gen_project hook script didn't exit successfully
2024-05-04 12:33:06,028 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2024-05-04 12:33:06,037 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2024-05-04 12:33:06,039 | Sending Telemetry: {'metrics': [{'commandRun': {'requestId': '55d97f37-4520-4617-a247-2a3567a20d54',
'installationId': 'bd80cbf9-f65d-4e2e-9994-227fa828c739', 'sessionId': '8abc2f0e-e1c8-4006-b545-01ae3f9c58c9', 'executionEnvironment':
'CLI', 'ci': False, 'pyversion': '3.11.8', 'samcliVersion': '1.116.0', 'awsProfileProvided': False, 'debugFlagProvided': True,
'region': '', 'commandName': 'sam init', 'metricSpecificAttributes': {'projectType': 'CFN'}, 'duration': 14093, 'exitReason':
'GenerateProjectFailedError', 'exitCode': 1}}]}
2024-05-04 12:33:06,039 | Unable to find Click Context for getting session_id.
2024-05-04 12:33:06,056 | Sending Telemetry: {'metrics': [{'events': {'requestId': '1d749600-69ae-477d-b6e4-d4f042161188',
'installationId': 'bd80cbf9-f65d-4e2e-9994-227fa828c739', 'sessionId': '8abc2f0e-e1c8-4006-b545-01ae3f9c58c9', 'executionEnvironment':
'CLI', 'ci': False, 'pyversion': '3.11.8', 'samcliVersion': '1.116.0', 'commandName': 'sam init', 'metricSpecificAttributes':
{'events': [{'event_name': 'SamConfigFileExtension', 'event_value': '.toml', 'thread_id': '6aca96ee57164d33b22d768e6372370a',
'time_stamp': '2024-05-04 10:32:51.926', 'exception_name': None}, {'event_name': 'SamConfigFileExtension', 'event_value': '.toml',
'thread_id': 'f7318f23f60b49fe9830e2808ae7407e', 'time_stamp': '2024-05-04 10:32:51.934', 'exception_name': None}]}}}]}
2024-05-04 12:33:07,293 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out.
(read timeout=0.1)
2024-05-04 12:33:07,324 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out.
(read timeout=0.1)
Error: An error occurred while generating this project : Hook script failed (exit status: 1)

Update instructions to use sam build

sam build is a new command that eases build source code to the format expected by the Lambda runtime.

This issue is to reflect the work involved in doing so and whether it's sufficient to replace the experimental Makefile altogether.

Convert dummy example into a concrete example

Ideas to pick from when time allows:

• Simple TODO API
• Guestbook
• Simple ATM

Add DOCKER flag to make build

I came across an issue where using docker build was essential for building psycopg2. I did some edits to the makefile to allow Docker container builds so I got the proper .so file for the lib. ex:

Maybe this is worth adding to the project? Let me know Ill open a pull req and do the proper testing fully.

package: _check_service_definition ##=> Builds package using Docker Lambda container
ifeq ($(DOCKER),1)
	$(info [*] Cleaning up local dev/builds before build task...)
	@$(MAKE) clean SERVICE="${SERVICE}"
	$(info [+] Packaging service '$(SERVICE)' using Docker Lambda -- This may take a while...)
	docker run -v $$PWD:/var/task -it lambci/lambda:build-python3.6 /bin/bash -c 'make _package SERVICE="${SERVICE}"'
else
	$(info [*] Cleaning up local builds before build task...)
	@$(MAKE) clean SERVICE="${SERVICE}"
	$(info [+] Packaging service '$(SERVICE)' -- This may take a while...)
	@$(MAKE) _package SERVICE="${SERVICE}"
endif

build: _check_service_definition _clone_service_to_build ##=> Same as package except that we don't create a ZIP
ifeq ($(DOCKER),1)
	$(info [+] Packaging service '$(SERVICE)' using Docker Lambda -- This may take a while...)
	docker run -v $$PWD:/var/task -it lambci/lambda:build-python3.6 /bin/bash -c 'make _install_deps SERVICE="${SERVICE}"'
else
	@$(MAKE) _install_deps SERVICE="${SERVICE}"
endif

Unable to import module 'app'

Unable to import module 'app'

Projects built with sam init --location gh:aws-samples/cookiecutter-aws-sam-python fail when deployed with the message:

• Python 2.7

Unable to import module 'app': cannot import name LifoQueue

• Python 3.6

Unable to import module 'app': cannot import name queue

Steps to reproduce

1. Download and install sam cli: pip install aws-sam-cli

2. Initialize a project: sam init --location gh:aws-samples/cookiecutter-aws-sam-python

3. When prompted by the command in step 2. enter these details

       [+] Initializing project structure...
       project_name [Name of the project]: foo
       project_short_description [A short description of the project]: test
       include_apigw [y]: n
       include_xray [y]:
       include_safe_deployment [y]:
       include_experimental_make [n]: y

4. Bundle and deploy

   $ make package SERVICE=ActionTakenEmail DOCKER=1
   $ aws cloudformation package --template-file template.yaml --output-template-file packaged.yaml --s3-bucket <your_bucket>
   $ aws cloudformation deploy --template-file $PWD/packaged.yaml --stack-name foo
   

5. Test via lambda console, using the default hello world test event should be enough.

The end result should be the error message listed above.

Now your Pipfile should look similar to:

[[source]]

url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"


[packages]

boto3 = "*"
aws-xray-sdk = "*"


[dev-packages]

pytest = "*"
pytest-cov = "*"

Now, If you remove the section from the Pipfile:

boto3 = "*"
aws-xray-sdk = "*"

Re-run steps 4 & 5 from above, it should now work.

Other relevant info:

 johnt $ make install                                                                                                                                                                                                                      [*] Install required packages...
Spawning environment shell (/bin/zsh). Use 'exit' to leave.
. /Users/johnt/.local/share/virtualenvs/foo-ju0e2WaU/bin/activate
Agent pid 5041
 johnt $ . /Users/johnt/.local/share/virtualenvs/foo-ju0e2WaU/bin/activate                                                                                                                                                                  johnt $ python --version                                                                                                                                                                                               Python 2.7.15
 johnt $ which python                                                                                                                                                                                                   /Users/johnt/.local/share/virtualenvs/foo-ju0e2WaU/bin/python
 johnt $ sam --version
SAM CLI, version 0.4.0

Also tested with SAM CLI, version 0.3.0

[Discussion] Adding a pipeline for this example

Proposal: Add CI/CD Pipeline for this template

Last updated: June 6th

Abstract

This proposal aims to add an additional capability to include a CI/CD Pipeline after an user completes its first deployment, so that it becomes a progressive experience after getting used to the programming and deployment model without having to enforce that a Pipeline should be used from Day 1.

Background

When customers complete the steps outlined in the README upon the project generation they expressed they'd like to progress in their experience and understand how to create a Pipeline that will help them "productionize" their Serverless App, however they may not have experience with all services required to do so.

Proposal

I propose 2 methods to implement this that wouldn't affect the current experience this template provides:

1. Develop a CI/CD pipeline in a separate Github repo that could be pulled into an existing Python project
2. Include a pipeline option in cookiecutter.json and if chosen buildspec.yaml and pipeline.yaml would be added upon project generation

Here's how the Developer Experience would look like for method 1:

1. Initialize a new app: sam init --runtime python or sam init --location gh:aws-samples/cookiecutter-aws-sam-python
2. Work on business logic and deploy their service without any pipeline
3. Once they feel comfortable with the environment they can add generate a 3-stage CI/CD pipeline with sam init --location gh:aws-samples/cookiecutter-aws-sam-pipeline
4. This would add 3 files to their current project folder: buildspec.yaml (if not already created), pipeline.yaml and Pipeline-Instructions.MD
   • buildspec.yaml would contain comments on how to create a build step (e.g install deps here, run unit tests her, etc.) including the sam package command that will work for the pipeline
   • pipeline.yaml would contain a Cloudformation Template that has everything they'd need to generate this CI/CD pipeline including comments on how to extend or understand options used
   • Pipeline-Instructions.MD would contain information about
     • How the Pipeline is constructed
     • Any changes that may be required to customize the experience (e.g. Single Account vs Multi-Account)
     • How to create a Cloudformation stack to generate this pipeline
     • How to push their code to the new source code repository

Here's how the Developer Experience would look like for method 2:

1. Initialize a new app: sam init --runtime python or sam init --location gh:aws-samples/cookiecutter-aws-sam-python
2. Work on business logic and deploy their service without any pipeline
3. buildspec.yaml and pipeline.yaml would be included upon project generation and already customized with defaults for the chosen runtime
4. README.MD would be extended to include details on how to generate this CI/CD pipeline under Appendix

Rationale

Each method has its pros and cons and this aims to outline them to aid with a decision alongside the comments by the community too.

Develop a CI/CD pipeline in a separate Github repo

Pros:

• Clean start and separation as pipeline can have different contributors
• One could fork and create a pipeline for any Runtime with their own defaults in the future with subtle changes
• Can add a pipeline to an existing project without impacting it

Cons:

• One needs to know of this template's existence otherwise it wouldn't know how to use it

Include a pipeline option upon project generation

Pros:

• Single repo and easier to correlate target project for this pipeline
• Forking this repository for other runtimes would include the entire package idea (project structure, tests and pipeline)

Cons:

• Customers would need to re-initialize this project only to have access to the pipeline feature

Compatibility

Option 1 doesn't break compatibility whilst the second would only affect existing customers who already initialized a project with this template.

Implementation

Implementation for either methods would be similar so here are the common parts:

• Create a pipeline.yaml that will implement a 3-stage CI/CD CodePipeline pipeline that will include:
  • CodeCommit or Github as a source code repository
  • CodeBuild for running tests, packaging and any security tests
  • Cloudformation to deploy the SAM stack
  • S3 Bucket for storing Build Artifacts
  • Single or Multi-Account option so an user can have placeholders and an explanation as to how to use that in a multi-account scenario
• Create a buildspec.yaml that will implement the build steps for CodeBuild that will include:
  • Run all tests with a coverage baseline
  • Packaging the Python project with one or more functions
  • Create a SAM package
• Append Pipeline into Appendix to include the following instructions:
  • How to create a pipeline stack
  • How to initialize a repo to connect git changes to this newly created pipeline
  • How to add Pipeline metrics (# of deployments, lead time, etc.)
• Extend hooks to not overwrite any existing buildspec.yaml

For the second approach the following would be needed in addition to the above:

• Extend cookiecutter.json and add a include_pipeline option and default to n
• Extend logic in hooks to clean up buildspec.yaml and pipeline.yaml if include_pipeline isn't chosen

make test results in no module named pytest

When running make test it fails due to missing pytest. It is in the Pipfile and lock.. It seems as if make install isn't actually installing anything based on the output from the virtualenv/lib/python3.6/site-packages/

Fix up bootstrapping and update tests

• Update Makefile instructions to use make dev
• Update tests to incorporate changes in Powertools docs
• Pin major semantic version

Exemplify multiple files

• Separate handler from business logic
• Use a child Logger

[Discussion] Convert Makefile into a Python script

Proposal: Convert Makefile into a Python script

Last updated: June 6th

Abstract

As of today, OSX and Linux users are able to have hot-reloading thanks to Makefile logic that creates hard-links on a given service/folder, however this is currently not possible for Windows due to Operating Systems differences.

Windows hard links seem to be supported when using Python and this proposal aims to convert this Makefile to make these features more accessible to everyone. Additionally, this could open the opportunity to add other mechanisms such as deployment asked here #2 and make the packaging feature more explicit.

Background

When Windows customers try to use Makefile for hot-reloading or easier Python packaging they need to install Git Bash or Windows Sub-system in order to get access to UNIX tools to make this work.

Linux and OSX users also expressed they'd like to have requirements.txt or Pipfile per function and not at the project level as well as a deploy target, of which would add complexity to Makefile and error prone not to mention that is difficult to test Makefile changes today.

Proposal

I propose porting Makefile essential features to a Python CLI script that would include:

• build
• package
• run
• test
• clean

Rationale

By implementing this Python CLI script we could also give more options to customers and be less opinionated as to where their dependencies should be declared and more importantly allow for testing this functionality as Makefile is only for advanced users.

Difficulty here is deciding on a name for this Python CLI script because setup.py is widely used for Python Packaging with setuptools and manage.py is used for Django Apps so perhaps bootstrap.py could fit the bill here.

Also needs some thinking whether it makes sense to use only requirements.txt to simplify initial design or to support both Pipfile and requirements.txt.

Compatibility

No breaking change to the application itself

Implementation

This could be implemented using Python Click or pure Python Argparse to avoid additional dependencies:

• Move test folder inside the function
• Create a test folder to explicitly have tests for this Python CLI script
• Add the following targets
  • build
    • Install deps and creates structure for hot-reloading without having to build a wrapper or anything
    • Define parameter for Source Code directory and optional for requirements.txt/Pipfile
  • package
    • Install deps and builds a Lambda compliant ZIP package and optionally with LambCI Docker Lambda Runtime when needed
    • Define parameter to allow straight up zip instead of installing deps in case they are already ready
  • run
    • Runs sam local start-api and supports for running Lambda containers within a Docker network
  • test
    • Runs pytest and set XRAY env to run that locally
    • Define parameter for specifying tests folder
• clean
  • Deletes zip file and build folder

Use the new Powertools API Gateway event handler

https://awslabs.github.io/aws-lambda-powertools-python/latest/core/event_handler/api_gateway/

Rebuild project with latest SAM build, and Lambda powertools

Serverless ecosystem evolved a lot since we started this template. We've started Lambda Powertools project with Python being the first language support - Powertools embrace many best practices and developer experience enhancements that we worked around here a few years ago.

This issue is to track a revamp to:

• Use Lambda Powertools - Tracer, Logger for structured logging, and Metrics for custom metrics
• Use Python linters, formatters, security baseline, and other good practices by default
• Use Poetry now that's 1.0
• Use SAM build and suggest VS Code extension as opposed to OS hard link hacks

Update project to target latest stable: Python 3.8

Python 3.8 support has been released for AWS Lambda. All dependent images and services have been updated to accommodate Python 3.8 on AWS Lambda, so there isn't much reason to not target new projects for 3.8.

Apply opinionated style & conventions

As of now, we're primarily sharing a dummy project with Powertools and its opinions, but the project itself doesn't have code style, and tooling to help enforce so.

• Bring in code formatter
• Bring in code linting
• Bring in code complexity baseline (cyclomatic)

Add support for sam invoke to Makefile

Add support for sam invoke to Makefile

Update the Makefile to include support for make invoke

Context

There may be a time where a developer wishes to test functions that are not linked to an API gateway facade. It may be desireable to simply develop and run make invoke SERVICE=first_function EVENT=first_function_event.json a part of development workflow.

Problem statement

The current Makefile implementation assumes that functions developed will be wrapped by API gateway.

Proposed Solution

Add a make target that enables support for make invoke

Create functional cookiecutter tests

• Render this project to ensure it introduces no regression
• Run project's bootstrapping
• Run project's tests