globaldatanet/cost-reporter

AWS Cost Reporter for Slack

Cost Reporter

A lambda which sends a daily cost and trend report to slack. It helps with one of the three FinOps phases: Inform. It can send you a report every day / only when cost increases / only when cost breached a threshold depending on your configuration.

Helpful to keep an eye in new projects when the architecture changes quickly, but also for existing projects if you want to keep a close eye on cost.

Example:

Configuration

The following settings can be configured:

• Title: The title for the cost report (i.e.: "Project: XYZ")
• Days: The days to report (i.e. 10)
• MinDailyCost: The minimal daily cost required to trigger a report in $ (i.e. 10)
• OnlyNotifyOnIncrease: Whether to send a report only when the cost increased from yesterday to today
• TargetChannel: The target channel (i.e. "#costoptimization")

Deployment

Prerequisites:

• AWS SAM CLI
• A slack token for the lambda (xoxx-....) with the permission to write files and send messages
• A SecureString SSM Parameter for the Slack token: /cost-reporter/slack-token

How to deploy:

sam build --use-container
sam deploy --guided

cost-reporter's Issues

Implement black as code formatter

Using a code formatter like black https://github.com/psf/black makes it easier to work on the project with different IDEs. There is also a mixed usage of single and double quotes, which a formatter will fix for you.

GitHub Actions for Running Tests

A CI task that runs the project's tests before any push event.

Fix and improve flake8 findings

Running flake8 cost-reporter results in

cost-reporter/tests/test_lambda_function.py:39:63: W292 no newline at end of file
cost-reporter/src/stacked_bar.py:34:19: F821 undefined name 'previous_bar'
cost-reporter/src/stacked_bar.py:35:37: F821 undefined name 'previous_bar'
cost-reporter/src/stacked_bar.py:39:9: F841 local variable 'previous_bar' is assigned to but never used

Also remove the ignored findings in the .flake8 file

cost-reporter/.flake8

Line 3 in f15d3fd

ignore = E221, E251, E266

This will result in

cost-reporter/src/slack_sender.py:19:70: E251 unexpected spaces around keyword / parameter equals
cost-reporter/src/slack_sender.py:19:72: E251 unexpected spaces around keyword / parameter equals

ClientError: An error occurred (AccessDeniedException) when calling the GetParameter operation

Improve SAM template

1. Remove the NoEcho property, because it is no secret, which SSM Parameter you use. Also add the Description property to explain, what it does.
   

   cost-reporter/template.yaml

   Lines 10 to 13 in f15d3fd

   	SlackTokenParameterName:
   	Type: String
   	Default: "/cost-reporter/slack-token"
   	NoEcho: true

2. Those types are Number
   

   cost-reporter/template.yaml

   Lines 18 to 25 in f15d3fd

   	Days:
   	Type: String
   	Default: 10
   	Description: "The amount of days to include in the report"
   	MinDailyCost:
   	Type: String
   	Default: 0
   	Description: "The minimal daily cost to trigger a report"

3. If this parameter is supposed to be of type boolean, then also enforce it
   

   cost-reporter/template.yaml

   Lines 26 to 29 in f15d3fd

   	OnlyNotifyOnIncrease:
   	Type: String
   	Default: true
   	Description: "Whether to only notify if cost increased within the last day"

  OnlyNotifyOnIncrease:
    Type: String
    AllowedValues: 
      - "true"
      - "false"
    Default: "true"
    Description: "Whether to only notify if cost increased within the last day"

4. Instead of adding the logs permissions
   

   cost-reporter/template.yaml

   Lines 85 to 92 in f15d3fd

   	-
   	Sid: "Logging"
   	Effect: "Allow"
   	Action:
   	- "logs:CreateLogGroup"
   	- "logs:CreateLogStream"
   	- "logs:PutLogEvents"
   	Resource: !Sub "arn:aws:logs:${AWS::Region}:${AWS::AccountId}:*"

   
   just assign the AWS managed policy arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

Readme needs improvment

The readme is missing how to create a slack token and which permission the token needs.

Add type hinting

Add type hinting in the whole project and activate mypy. http://mypy-lang.org/

Using A Test Runner

Simply use a test runner to access the project-level tests easier and implement the #15 issue easier.

Improve slack_sender.py

Since the project uses Python 3.8 it is recommend to use pathlib instead of os.path, because of the support for different OS filesystems.
https://docs.python.org/3/library/pathlib.html

cost-reporter/cost-reporter/src/slack_sender.py

Line 16 in f15d3fd

filepath = os.path.join(os.path.dirname(os.path.realpath(__file__)), filename)

Improve stacked_bar.py

1. By PEP8 convention constants are written in all capital letters
   

   cost-reporter/cost-reporter/src/stacked_bar.py

   Lines 9 to 15 in f15d3fd

   	yellow = "#fed919"
   	cyan = "#2db2a1"
   	pink = "#ff6f61"
   	magenta = "#a41d53"
   	purple = "#6a45ce"
   	light_green = "#3ecc54"
   	dark_green = "#2db2a1"

2. To save time with list conversion this
   

   cost-reporter/cost-reporter/src/stacked_bar.py

   Lines 20 to 21 in f15d3fd

   	first_key = list(all_bars.keys())[0]
   	days = len(all_bars[first_key])

   
   can be written as

days = next(iter(all_bars.values()))

Slack token Parameter is not created in SSM

Right now the user needs to create the slack token parameter manually - would be nice if the Parameter would be created via CFN.

Improve lambda_function.py

1. this could be written as a liste comprehension
   

   cost-reporter/cost-reporter/src/lambda_function.py

   Lines 16 to 19 in f15d3fd

   	dates = []
   	for i in range(0, days):
   	day = (datetime.today() - timedelta(days=i)).strftime("%d")
   	dates.append(day)

2. the parentheses are not needed here
   

   cost-reporter/cost-reporter/src/lambda_function.py

   Line 57 in f15d3fd

   return (daily_cost, dates)