terraform-aws-modules/terraform-aws-managed-service-grafana

Terraform module to create AWS Managed Service for Grafana (AMG) resources 🇺🇦

terraform-aws-managed-service-grafana's Issues

Network Access Control requiring vpce_ids

Description

Adding a network_access_control attribute requires specifying vpce_ids while from the AWS Console it is possible to use a managed prefix list only.

• [v] ✋ I have searched the open/closed issues and my issue is not listed.

Versions

• Module version [Required]: Latest
• Terraform version: v1.6.4
• Provider version(s): 5.39.1

Reproduction Code [Required]

`resource "aws_grafana_workspace" "main" {
account_access_type = "CURRENT_ACCOUNT"
authentication_providers = ["AWS_SSO"]
[...]
grafana_version = "9.4"

network_access_control {
prefix_list_ids = var.prefix_list_ids
}`

Expected behavior

Network access control configured with the specified Managed prefix list only.

Equivalent to
aws grafana update-workspace --workspace-id g-xxx --network-access-control "{\"prefixListIds\": [\"pl-xxx\"], \"vpceIds\":[]}"

Actual behavior

Error: Missing required argument │ │ on ..\..\modules\grafana\main.tf line 10, in resource "aws_grafana_workspace" "main": │ 10: network_access_control { │ │ The argument "vpce_ids" is required, but no definition was found.

Add new variable create_saml to be able to skip saml config

Is your request related to a problem? Please describe.

I need a way to create the workspace, with SAML specify in authentication_providers but skipping aws_grafana_workspace_saml_configuration

The order of my scenario is like this

1. create aws_grafana_workspace_saml_configuration with authentication_providers = ["SAML"]
2. get the output workspace_endpoint
3. Deploy Keycloak with input workspace_endpoint (helm)
4. Get from Keycloak output the url for idp_metadata_url
5. Create aws_grafana_workspace_saml_configuration using output workspace_id

Describe the solution you'd like.

New variable create_saml so I can set to false

Describe alternatives you've considered.

Don't have any other

Additional context

The scenario is for EKS Blueprints GitOps

Broken SSO setup

Description

I am constantly hitting a: Error: creating Grafana Workspace: ValidationException: SSO is not enabled in any region. when creating a new AMG instance using the Terraform example shown below.

• [ x ] ✋ I have searched the open/closed issues and my issue is not listed.

Versions

• Module version [Required]: 2.1.1
• Terraform version: 1.5.7
• Provider version(s): 5 something, latest

Reproduction Code [Required]

Make sure you setup your AWS_ variables.

terraform init && terraform plan

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 5"
    }
  }
}

provider "aws" {
  region = "eu-central-1"
}

locals {
  name = "my-own-grafana"
}

module "managed_grafana" {
  source  = "terraform-aws-modules/managed-service-grafana/aws"
  version = "~> 2"

  name                      = local.name
  account_access_type       = "CURRENT_ACCOUNT"
  authentication_providers  = ["AWS_SSO"]
  permission_type           = "SERVICE_MANAGED"
  data_sources              = ["PROMETHEUS"]
  notification_destinations = ["SNS"]
  grafana_version           = "9.4"

  # Workspace API keys
  workspace_api_keys = {
    admin = {
      key_name        = "admin"
      key_role        = "ADMIN"
      seconds_to_live = 3600
    }
  }
}

Expected behavior

A working AMG.

Actual behavior

Error due to missing SSO, even after enabling the AWS Identity Center in the UI for the same region the AMG is created in.

Additional context

Not even using the underlying AWS provider resources. directly helped.

Way to expose resources attributes in a data to allow grafana and prometheus providers to work nicely

• Yes ✅: managed grafana and managed prometheus

Is your request related to a problem? Please describe.

When trying to provision grafana using grafana provider I cannot do it in the same terraform run, the common provider issue of resource not being known at the creation time for provider to refresh state, or being deleted on a destroy which results in some resources being orphaned in the terraform state making it invalid.

Describe the solution you'd like.

Right now both prometheus workspace data type, and grafana workspace data type are being found by id. ID is not in the outputs. It would also be preferable to be able to find those by { Tag = Name } rather then just by the id which allows other terraform runs to get those without creating a file just to pass id in between or read it directly from terraform stat (sic!)

Except for being able to find a resource using data query by name it would also be preferable if grafana api keys would be in the attributes as well. Meaning other terraform run can get those staticly in the provider definition.

Something like this:

data "aws_grafana_workspace" "example" {
  filter {
    name   = "name"
    values = ["$MYENV-GRAFANA"]
}

provider "grafana" {
  url  = "https://${data.aws_grafana_workspace.example.endpoint}"
  auth = data.aws_grafana_workspace.example.auth_keys.admin.json
}

Describe alternatives you've considered.

Additional context

Add rotate for API keys

Issue

The API keys created with the input workspace_api_keys are not rotated when they expire.

Describe the solution you'd like.

The keys should be replaced automatically at the next apply.

Possible solution

I found a way to achieve this here:
hashicorp/terraform-provider-aws#27043 (comment)

locals {
  expiration_days    = 30
  expiration_seconds = 60 * 60 * 24 * local.expiration_days
}

resource "time_rotating" "rotate" {
  rotation_days = local.expiration_days
}

resource "time_static" "rotate" {
  rfc3339 = time_rotating.rotate.rfc3339
}

resource "aws_grafana_workspace_api_key" "admin_key" {
  key_name        = # ...
  key_role        = # ...
  seconds_to_live = local.expiration_seconds
  workspace_id    = # ...

  lifecycle {
    replace_triggered_by = [
      time_static.rotate
    ]
  }
}

Add network_access_control as an optional configuration to the module to create workspace with restricted access

Is your request related to a new offering from AWS?

• No

Is this functionality available in the AWS provider for Terraform? See CHANGELOG.md, too.

• Yes ✅: please list the AWS provider version which introduced this functionality
  aws provider 4.58.0. https://registry.terraform.io/providers/hashicorp/aws/4.58.0/docs/resources/grafana_workspace

Is your request related to a problem? Please describe.

• No, its not a problem , its just that currently the module will always create a workspace with open public access , and I have to manuallly assign the vpc endpoint I have created to it.

Describe the solution you'd like.

I saw that the resource grafana_workspace can optionaly accept network_access_control and I was wondering if this can be requested as a feature request to the managed_grafana_module to accept it optionally so that I can initialize the workspace with restricted access instead of manually changing it.

Describe alternatives you've considered.

I have been tried only one alternative solution and that is to assigning a VPC endpoint manually to the grafana workspace after I create it with terraform with open access.

Additional context

I have been playing around with the module and I came to the use case where I want to create the workspace with a vpc endpoint so that upon creation it is in a private subnet behind a VPN

redundant hyphen sign ?

terraform-aws-managed-service-grafana/main.tf

Line 70 in 14dc9a1

name_prefix = var.security_group_use_name_prefix ? "${local.security_group_name}-" : null

if this hyphen sign is in purpose ?
can i remove it ?

name_prefix = var.security_group_use_name_prefix ? "${local.security_group_name}-" : null

Adjust documentation to prevent errors while getting started

Description

I ran into a few errors while trying to perform an initial standup using this module. I think there are components of the documentation that could be improved to better describe how the module should be used, especially so that users do not run into a dead-end and meaningless API response from AWS, which is another problem that should be resolved not in this repository.

Required name input

Problem Description

One of the inputs "name" or "stack_set_name" is actually required, and must match certain syntax. The code sets this as null by default, which I think is fine for distribution, but the documentation should mention that you need to create this value since not creating it will cause an error in Terraform during planning:

Reproduction Code [Required]

module "managed-service-grafana" {
  source  = "terraform-aws-modules/managed-service-grafana/aws"
  version = "1.6.0"
  data_sources = ["PROMETHEUS"]
  iam_role_name = "AMG"
  associate_license = false
}

Terminal Output

╷
│ Error: Error in function call
│ 
│   on .terraform/modules/managed-service-grafana/main.tf line 27, in resource "aws_grafana_workspace" "this":
│   27:   stack_set_name            = coalesce(var.stack_set_name, var.name)
│     ├────────────────
│     │ while calling coalesce(vals...)
│     │ var.name is null
│     │ var.stack_set_name is null
│ 
│ Call to function "coalesce" failed: no non-null, non-empty-string arguments.

When providing a value for "name", but with bad syntax (this one with spaces in the string), Terraform will plan correctly, but AWS will return a fairly useless error:

Reproduction Code [Required]

module "managed-service-grafana" {
  source  = "terraform-aws-modules/managed-service-grafana/aws"
  version = "1.6.0"
  name = "Company Name with spaces"
  data_sources = ["PROMETHEUS"]
  iam_role_name = "AMG"
  associate_license = false
}

Terminal Output

╷
│ Error: error updating Grafana Workspace (g-27f2e365af): BadRequestException: 
│       status code: 400, request id: 96b32ea4-db72-4e9e-a8ad-c617d5a82a32
│ 
│   with module.managed-service-grafana.aws_grafana_workspace.this[0],
│   on .terraform/modules/managed-service-grafana/main.tf line 12, in resource "aws_grafana_workspace" "this":
│   12: resource "aws_grafana_workspace" "this" {
│ 
╵

Suggested fix

I agree with not calling these fields required, as either one or the other is required. The descriptions should instead be modified to indicate this:

• README.md > Inputs > name > Description = "The Grafana workspace name. Required if stack_set_name is not provided. Valid special characters include "-", ".", "_", “~”. Cannot contain non-ASCII characters or spaces. Max length of 255 characters."
• variables.tf > variable "name" > description = "The Grafana workspace name. Required if stack_set_name is not provided. Valid special characters include "-", ".", "_", “~”. Cannot contain non-ASCII characters or spaces. Max length of 255 characters."
• README.md > Inputs > stack_set_name > Description = "The AWS CloudFormation stack set name that provisions IAM roles to be used by the workspace. Required if name is not provided. Valid special characters include "-", ".", "_", “~”. Cannot contain non-ASCII characters or spaces. Max length of 255 characters."
• variables.tf > variable "stack_set_name" > description = "The AWS CloudFormation stack set name that provisions IAM roles to be used by the workspace. Required if name is not provided. Valid special characters include "-", ".", "_", “~”. Cannot contain non-ASCII characters or spaces. Max length of 255 characters."

I extracted the last 3 sentences from the setup process for AMG in the console, which I don't think every person who uses this module should have to visit in order to troubleshoot, so it would be ideal if it is in the descriptions for each input.

Unlicensed Grafana

Problem Description

Grafana is freely distributed, open source software, yet the configuration of this module as it's currently configured will error and fail to properly build the license_association resource. Easy enough to troubleshoot, but an unnecessary error for new users to encounter.

Reproduction Code [Required]

module "managed-service-grafana" {
  source  = "terraform-aws-modules/managed-service-grafana/aws"
  version = "1.6.0"
  name = "Company_Name_with_underscores"
  data_sources = ["PROMETHEUS"]
  iam_role_name = "AMG"
}

Terminal Output

╷
│ Error: error creating Grafana License Association: ValidationException: Active marketplace agreement not found
│ {
│   RespMetadata: {
│     StatusCode: 400,
│     RequestID: "a343e176-014d-4299-8cfb-ae275350eb0b"
│   },
│   Message_: "Active marketplace agreement not found"
│ }
│ 
│   with module.managed-service-grafana.aws_grafana_license_association.this[0],
│   on .terraform/modules/managed-service-grafana/main.tf line 293, in resource "aws_grafana_license_association" "this":
│  293: resource "aws_grafana_license_association" "this" {
│ 

Suggested fix

I recommend that the variable associate_license be set to false as a default so that the system will build regardless of whether the user has a license attached to the account. The user can easily add a new attribute to the module call which enables the license association if they have one.
Regardless of whether we agree on that point, I think more description should be added to this variable description to make clear how the variable should be used.

• variables.tf > variable "associate_license" > default = false
• variables.tf > variable "associate_license" > description = "Determines whether a license will be associated with the workspace. Use false if you do not have an active enterprise license in the AWS Marketplace."
• README.md > Inputs > associate_license > Description = "Determines whether a license will be associated with the workspace. Use false if you do not have an active enterprise license in the AWS Marketplace."

• ✋ I have searched the open/closed issues and my issue is not listed.

⚠️ Note

Before you submit an issue, please perform the following first:

1. Remove the local .terraform directory (! ONLY if state is stored remotely, which hopefully you are following that best practice!): rm -rf .terraform/
2. Re-initialize the project root to pull down modules: terraform init
3. Re-attempt your terraform plan or apply and check if the issue still persists

Versions

• Module version [Required]: 1.6.0

• Terraform version: 1.3.6

• Provider version(s): registry.terraform.io/hashicorp/aws v4.45.0

Reproduction Code [Required]

Steps to reproduce the behavior:

I am not using workspaces.

N/A regarding cache, as I am suggesting documentation changes.

I describe the steps taken in each problem description above. Sorry for the strange formatting, I'd rather keep all of the details logically grouped.

Expected behavior

I describe the problem description in each problem description above. Sorry for the strange formatting, I'd rather keep all of the details logically grouped.

Actual behavior

I describe the symptom in each problem description above. Sorry for the strange formatting, I'd rather keep all of the details logically grouped.

Terminal Output Screenshot(s)

I provide the terminal output in each Terminal Output section above. Sorry for the strange formatting, I'd rather keep all of the details logically grouped.

Additional context

edit > added README entry to match var.associate_license

Defaults for IAM permissions are inconsistent in variables and examples

Description

With AWS Managed Grafana, you can choose between "Service Managed" or "Customer Managed" IAM permissions.
Either AWS creates IAM roles/policies for you or you supply your own.

Code logic improvement
Now, the code has conditional creation of IAM resources

terraform-aws-managed-service-grafana/main.tf

Lines 36 to 37 in bd896b8

	create_account_policy = local.create_role && var.account_access_type == "CURRENT_ACCOUNT"
	create_custom_policy = length(setintersection(var.data_sources, ["CLOUDWATCH", "AMAZON_OPENSEARCH_SERVICE", "PROMETHEUS", "SNS"])) > 0

adding a sanity check ..&& var.permission_type == "CUSTOMER_MANAGED could make sense imho

Variable defaults issue
The defaults are mismatching and dont make sense in that regard:

terraform-aws-managed-service-grafana/variables.tf

Lines 41 to 44 in bd896b8

	variable "permission_type" {
	description = "The permission type of the workspace. If `SERVICE_MANAGED` is specified, the IAM roles and IAM policy attachments are generated automatically. If `CUSTOMER_MANAGED` is specified, the IAM roles and IAM policy attachments will not be created"
	type = string
	default = "SERVICE_MANAGED"

and

terraform-aws-managed-service-grafana/variables.tf

Lines 81 to 84 in bd896b8

	variable "create_iam_role" {
	description = "Determines whether a an IAM role is created or to use an existing IAM role"
	type = bool
	default = true

defaults should either favor customer-managed or aws-managed creation. the business logic results in favoring customer-managed while the default of permission_type suggests differently

Examples
The "complete" example is misleading, as it again has create_iam_role=true together with permission_type="SERVICE_MANAGED". While demonstrating all possible configurations, they still should make sense as some people copy blindly.

(Also the example in README has same flaw as it shows permission_type="SERVICE_MANAGED" together with create_iam_role=true from defaults.

Module should not default to an enterprise licence

Is your request related to a new offering from AWS?

Is this functionality available in the AWS provider for Terraform? See CHANGELOG.md, too.

• Yes ✅: please list the AWS provider version which introduced this functionality
• Current version

Is your request related to a problem? Please describe.

The aws_grafana_license_association defaults to ENTERPRISE. The enterprise licence has a large cost involved, I'm not sure if this will make the marketplace purchase but could inadvertently cause a cost spike.

If possible it would be great to have the option to disable the aws_grafana_license_association resource on its own.

Describe the solution you'd like.

Switch default from ENTERPRISE to ENTERPRISE_FREE_TRIAL and provide a variable to disable aws_grafana_license_association

Describe alternatives you've considered.

Will have to avoid using this module to not use the aws_grafana_license_association resource

Additional context

Grafana API key or a way to provision dashboards

Hi, I wonder is there a way to get an output of grafana api key or provision dashboards via json files while creating a grafana? Helm/Kubernetes created grafana has ways for it, but I cannot seem to find a way to do it via terraform with aws managed grafana. If there is no way to do it except for manual UI, then please do tell and sorry for bothering.

Hardcoded policy's to attach have incorrect paths

Description

Error: Error attaching policy arn:aws:iam::aws:policy/AmazonGrafanaAthenaAccess to IAM Role. NoSuchEntity: Policy arn:aws:iam::aws:policy/AmazonGrafanaAthenaAccess does not exist or is not attachable. status code: 404

• ✋ I have searched the open/closed issues and my issue is not listed.

Versions

• Module version [Required]:
• v1.3.0
• Terraform version:
  v1.1.7
• Provider version(s):
  hashicorp/aws v4.18.0

Reproduction Code [Required]

module "managed_grafana" {
  source  = "terraform-aws-modules/managed-service-grafana/aws"
  version = "v1.3.0"

  # Workspace
  name                     = "grafana-${local.cluster_name}"
  associate_license        = false
  description              = "AWS Managed Grafana for ${local.cluster_name}"
  account_access_type      = "CURRENT_ACCOUNT"
  authentication_providers = ["SAML"]
  permission_type          = "SERVICE_MANAGED"
  data_sources = [
    "AMAZON_OPENSEARCH_SERVICE",
    "ATHENA",
    "CLOUDWATCH",
    "PROMETHEUS",
    "REDSHIFT",
    "SITEWISE",
    "TIMESTREAM",
    "XRAY"
  ]
  notification_destinations = ["SNS"]
  stack_set_name            = "grafana-${local.cluster_name}"

  # Workspace IAM role
  create_iam_role                = true
  iam_role_name                  = "${local.cluster_name}-managed-grafana-role"
  use_iam_role_name_prefix       = false
  iam_role_description           = "IAM Role for AWS Managed Grafana"
  iam_role_path                  = "/grafana/"
  iam_role_force_detach_policies = true
  iam_role_max_session_duration  = 7200

  # Workspace SAML configuration
  saml_admin_role_values  = ["DevOps"]
  saml_editor_role_values = ["editor"]
  saml_email_assertion    = "mail"
  saml_groups_assertion   = "groups"
  saml_login_assertion    = "mail"
  saml_name_assertion     = "displayName"
  saml_org_assertion      = "org"
  saml_role_assertion     = "DevOps"
  saml_idp_metadata_xml   = file("${path.module}/idp/dev.xml")
}

Expected behavior

Apply without error

Actual behavior

Error on apply

Additional context

It seems that Redshift and Athena have incorrect policy paths hardcoded.

It should be:
arn:aws:iam::aws:policy/service-role/AmazonGrafanaAthenaAccess and arn:aws:iam::aws:policy/service-role/AmazonGrafanaRedshiftAccess

Add input option for security_group_ids to README

In the VPC options in the main.tf, there appears to only show an option to add the subnet_id, but not the security_group

  # vpc configuration
  vpc_configuration = {
    subnet_ids = module.vpc.private_subnets
  }

The official resource docs for aws_grafana_workspace shows security_group_ids as an option:

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/grafana_workspace
[VPC Configuration])
[security_group_ids]- (Required) - The list of Amazon EC2 security group IDs attached to the Amazon VPC for your Grafana workspace to connect.
[subnet_ids]- (Required) - The list of Amazon EC2 subnet IDs created in the Amazon VPC for your Grafana workspace to connect.

In the README example and the main.tf and the variables.tf there is no option or suggestion to use security_group_ids which appeared to indicate that adding existing security_group_ids to this module wasn't possible.

I've tested this by adding security_group_ids to the vpc_configuration block and it has successfully added my existing security groups to the workspace.

Feature request to add this the README example or the inputs or main.tf to confirm this this available in the module

Var for grafana_version

Is your request related to a new offering from AWS?

Is this functionality available in the [AWS provider for Terraform]

• Yes ✅: please list the AWS provider version which introduced this functionality

Is your request related to a problem? Please describe.

The grafana_workspace resource has support grafana_version but it's not possible to set it up: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/grafana_workspace.html#grafana_version

Describe the solution you'd like.

Add var.grafana_version

Describe alternatives you've considered.

Additional context

This is required to upgrade workspace version: Specifies the version of Grafana to support in the new workspace. Supported values are 8.4 and 9.4. If not specified, defaults to 8.4. Upgrading the workspace version isn't supported, however it's possible to copy content from the old version to the new one using AWS official migration tool.