[Feature Request] Support ECS/Fargate tasks for built-in dimensions

[aws]

[Feature Request] Support ECS/Fargate tasks for built-in dimensions

[otterley]

Similarly, the Agent should be able to substitute the ECS task ID into log stream names by querying the ECS task metadata endpoint. Right now only instance ID, hostname, local_hostname, and ip_address are supported. Best practice is not to allow ECS tasks to access EC2 instance metadata, which prevents the Agent from collecting host-level metadata such as the instance ID.

[pingleig]

For prometheus, it's actually a minor change, we only need to update the export label logic in service discovery implementation to include task arn

amazon-cloudwatch-agent/internal/ecsservicediscovery/decoratedtask.go

Lines 133 to 140 in 5fe51db

	labels := make(map[string]string)
	addExporterLabels(labels, containerNameLabel, c.Name)
	addExporterLabels(labels, taskFamilyLabel, t.TaskDefinition.Family)
	revisionStr := fmt.Sprintf("%d", *t.TaskDefinition.Revision)
	addExporterLabels(labels, taskRevisionLabel, &revisionStr)
	addExporterLabels(labels, taskGroupLabel, t.Task.Group)
	addExporterLabels(labels, taskStartedbyLabel, t.Task.StartedBy)
	addExporterLabels(labels, taskLaunchTypeLabel, t.Task.LaunchType)

[haojhcwa]

Hi, thank you for your feedback, we will consider adding this feature in our future road map.

[ivan-moto]

This would be very nice to have.

[arlyon]

I am using cloudwatch-agent to export prometheus metrics to cloudwatch and, thanks to prometheus, the only uniquely identifying field is the "instance" field: the IP address of the prom endpoint the data was collected from. This is okay but inconvenient (at the very best it means having to do a lookup first). Having access to the task ID would be great.

[arlyon]

Testing changes now, will open a PR for (only) prometheus soon :)

[github-actions]

This issue was marked stale due to lack of activity.

[georgej-github]

Similarly, the Agent should be able to substitute the ECS task ID into log stream names by querying the ECS task metadata endpoint. Right now only instance ID, hostname, local_hostname, and ip_address are supported. Best practice is not to allow ECS tasks to access EC2 instance metadata, which prevents the Agent from collecting host-level metadata such as the instance ID.

+1 for this, would be great if the cloudwatch agent also supported injecting the ECS task ID into log stream names.

[matthewlynden]

+1 for this, we had to move our cloudwatch agent configuration to use {ip_address} instead of the taskId when migrating to Fargate.