gp42/aws_cloudwatch

Chef cookbook providing AWS Cloudwatch Agent resource

AWS Cloudwatch Cookbook

This cookbook installs and configures AWS CloudWatch Agent.

Check out my blog at ops42.org.

Platform Support

• Ubuntu 16.04, 18.04, 20.04
• Debian 9, 10, 11
• Centos 7
• Fedora 29
• Amazon Linux

Cookbook Dependencies

• Chef 13+
• gpg
• sudo

Usage

Place a dependency on the aws_cloudwatch cookbook in your cookbook's metadata.rb

depends 'aws_cloudwatch', '~> 2.0.0'

Then in a recipe:

aws_cloudwatch_agent 'default' do
  action      [:install, :configure, :restart]
  json_config 'amazon-cloudwatch-agent.json.erb'
end

json_config

Amazon CloudWatch Agent configuration file which defines which metrics/logs are collected. Place the amazon-cloudwatch-agent.json.erb file to templates directory. This is an agent configuration for metrics and logs collection. See AWS documentation for more information: Manually Create or Edit the CloudWatch Agent Configuration File

config

The configuration file is at /opt/aws/amazon-cloudwatch-agent/etc. See AWS Documentation for more info. Default file looks like this:

# This common-config is used to configure items used for both ssm and cloudwatch access

## Configuration for shared credential.
## Default credential strategy will be used if it is absent here:
##            Instance role is used for EC2 case by default.
##            AmazonCloudWatchAgent profile is used for onPremise case by default.
# [credentials]
#    shared_credential_profile = "{profile_name}"
#    shared_credential_file= "{file_name}"

## Configuration for proxy.
## System-wide environment-variable will be read if it is absent here.
## i.e. HTTP_PROXY/http_proxy; HTTPS_PROXY/https_proxy; NO_PROXY/no_proxy
## Note: system-wide environment-variable is not accessible when using ssm run-command.
## Absent in both here and environment-variable means no proxy will be used.
# [proxy]
#    http_proxy = "{http_url}"
#    https_proxy = "{https_url}"
#    no_proxy = "{domain}"

You can modify this configuration by overriding attributes:

default['aws_cloudwatch']['config']['params']['shared_credential_profile']
default['aws_cloudwatch']['config']['params']['shared_credential_file']
default['aws_cloudwatch']['config']['params']['http_proxy']
default['aws_cloudwatch']['config']['params']['https_proxy']
default['aws_cloudwatch']['config']['params']['no_proxy']

It is also possible to configure it with environment variables instead:

SHARED_CREDENTIAL_PROFILE
SHARED_CREDENTIAL_FILE
HTTP_PROXY
HTTPS_PROXY
NO_PROXY

If you want to provide your own template for the configuration file, then you need to supply it as a config parameter to the resource:

aws_cloudwatch_agent 'default' do
  action :install
  config <config_template.erb>
end

Resources overview

aws_cloudwatch_agent

The aws_cloudwatch_agent resource installs AWS Cloudwatch Agent.

Parameters

• action - Possible actions with the agent: :install, :configure, :remove, :start, :stop, :restart
• config - A template name for a custom test-config.toml file
• config_params - A hash with test-config.toml parameters
• json_config - A template name for an amazon-cloudwatch-agent.json file

Example

aws_cloudwatch_agent 'default' do
  action          [:install, :configure, :restart]
  json_config     'amazon-cloudwatch-agent.json.erb'
  config_params   :shared_credential_profile => 'test_profile',
                  :shared_credential_file => '/etc/test_credential_file',
                  :http_proxy => 'http://192.168.0.1',
                  :https_proxy => 'https://192.168.0.1',
                  :no_proxy => 'http://192.168.0.10'
end

Development

Dependencies

• Ruby (see .tool-versions)
• Ruby Bundler

Install gems with bundler:

bundle i

Testing

Create a branch for this cookbook. Create a PR. This will trigger a build on Travis CI. Make sure it succeeds. You can also use test kitchen with AWS. Make sure you modify it with your custom parameters like Security Group id and others.

# Docker
KITCHEN_YAML=.kitchen.dokken.yml kitchen verify default
# AWS (uses OS credentials)
KITCHEN_YAML=.kitchen.aws.yml kitchen verify default

aws_cloudwatch's Issues

config-translator helper now writes to stderr

It appears that aws has released an update of the cloudwatch rpms.
In the update, the config-translator now writes things to stderr that previously went to stdout, and the grep fails in your resource to look for a successful run.

This command is executed.

/opt/aws/amazon-cloudwatch-agent/bin/config-translator --input /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json --output /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml --mode auto --config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml)

and this is the output
2022/05/12 16:11:28 D! [EC2] Found active network interface I! Detected the instance is EC2 2022/05/12 16:11:28 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json ... Cannot access : lstat : no such file or directory2022/05/12 16:11:28 unable to scan config dir with error: lstat : no such file or directory 2022/05/12 16:11:28 I! Valid Json input schema. I! Detecting run_as_user... No csm configuration found. No log configuration found. Configuration validation first phase succeeded

but the Valid Json input schema is not going to stdout.

I ran those commands as route to get the output.

Chef has to run twice before agent sends logs

Using v1.0.0:

Currently, chef has to be run twice before the agent send logs to CloudWatch. This is due to the service overwriting the .json file when starting for the first time.

Ubuntu 18.04

Is there any plan to make this compatible with Ubuntu 18.04?

Windows support

This cookbook is ready to add Windows support, just needs:

• Windows gpg verifiation logic
• Install
• Uninstall

Action :start is not idempotent

The script resource for :start action has no guards, and thus executed each time chef-client runs. It should get not_if/only_if guard to execute only if needed.