cloudposse/terraform-aws-mq-broker

Terraform module for provisioning an AmazonMQ broker

terraform-aws-mq-broker's Issues

MQ Broker should be recreate when id for SG changed

Describe the Bug

╷
│ Error: updating MQ Broker (b-f9764a70-79ab-41de-ba60-2f722296ba50) security groups: BadRequestException: Changing security groups is not supported for RabbitMQ brokers.
│ {
│   RespMetadata: {
│     StatusCode: 400,
│     RequestID: "52d88252-71f0-4800-9c41-aa8455b174bf"
│   },
│   ErrorAttribute: "securityGroups",
│   Message_: "Changing security groups is not supported for RabbitMQ brokers."
│ }
│ 
│   with module.rabbitmq.aws_mq_broker.default[0],
│   on ../../modules/mq/main.tf line 89, in resource "aws_mq_broker" "default":
│   89: resource "aws_mq_broker" "default" {
│ 
╵

Expected Behavior

id for security group don't change, or recreate rabbitMQ broker whe security_id was change

Steps to Reproduce

1. Create rabbitmq
2. Create sq
3. add new SQ to rabbitmq by variable allowed_security_group_ids
4. apply changes

Screenshots

No response

Environment

• LINUX
• Terraform v1.3.6
• Module: 3.1.0

Additional Context

No response

module.this.id Error

Describe the Bug

I'm calling terraform-aws-mq-broker v3.1.0 from terragrunt and get the following error on a plan. It looks like the module.this.id has no value and fails the two tests. I read that these modules should be compatible with the terragrunt.

The main.tf line 91 seems to be the issue.
broker_name = module.this.id

I'm not sure why the error is being thrown and assume it's related to the fact that the id is only available after the module is applied? What's the fix or what am I doing wrong in trying to use cp modules with terragrunt?

│ Error: expected length of broker_name to be in the range (1 - 50), got
│
│ with module.mq_broker.aws_mq_broker.default[0],
│ on .terraform/modules/mq_broker/main.tf line 91, in resource "aws_mq_broker" "default":
│ 91: broker_name = module.this.id
│
╵
╷
│ Error: expected value of broker_name to match regular expression "^[0-9A-Za-z_-]+$", got
│
│ with module.mq_broker.aws_mq_broker.default[0],
│ on .terraform/modules/mq_broker/main.tf line 91, in resource "aws_mq_broker" "default":
│ 91: broker_name = module.this.id
│
╵
Releasing state lock. This may take a few moments...
ERRO[0055] Terraform invocation failed in /app/infrastructure-live/dev/ca-central-1/dev/messaging/rabbitmq/.terragrunt-cache/DzpSoeycWeukzz45-SRvPTXf7Qw/aYg6jvKMeoPdWBiN8E8O6YgHPKw prefix=[dev/ca-central-1/dev/messaging/rabbitmq]
ERRO[0055] 1 error occurred:
* [/app/infrastructure-live/dev/ca-central-1/dev/messaging/rabbitmq/.terragrunt-cache/DzpSoeycWeukzz45-SRvPTXf7Qw/aYg6jvKMeoPdWBiN8E8O6YgHPKw] exit status 1

Expected Behavior

Module applies without error

Steps to Reproduce

Use the module with terragrunt version v0.48.3 based on the examples/complete.

Screenshots

No response

Environment

terragrunt version v0.48.3
Terraform v1.5.4
on linux_amd64

Additional Context

No response

Add support for RabbitMQ

When trying the RabbitMQ option the code fails,

│ Error: expected engine_type to be one of [ACTIVEMQ], got RabbitMQ
│
│ with module.mq_broker.aws_mq_broker.default[0],
│ on .terraform/modules/mq_broker/main.tf line 91, in resource "aws_mq_broker" "default":
│ 91: engine_type = var.engine_type
│
╵

using engine_type=RabbitMQ

Use of deprecated argument `overwrite` on `aws_ssm_parameter`

Describe the Bug

The overwrite argument on aws_ssm_parameter has been deprecated:

│ Warning: Argument is deprecated
│ 
│   with module.mq.module.mq_broker.aws_ssm_parameter.mq_application_username[0],
│   on .terraform/modules/mq.mq_broker/main.tf line 74, in resource "aws_ssm_parameter" "mq_application_username":
│   74:   overwrite   = var.overwrite_ssm_parameter
│ 
│ this attribute has been deprecated

See docs for reference: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter#overwrite

Expected Behavior

No deprecation warnings are printed.

Steps to Reproduce

Use terraform-aws-mq-broker 3.1.0 with the aws provider 5.x.

Screenshots

No response

Environment

No response

Additional Context

No response

Module enabled toggle is not implemented in main and outputs files

In the documentation, it says we can use enabled to disable/enable the resource creation, however, it has not been fully implemented. Can someone please look into it?

ENH: Defining allowed port without traffic source returns no feedback

Describe the Feature

Additional feedback when allowed_ingress_ports argument is set, but no source SG or CIDR is set.

Expected Behavior

Error or Warning output that setting allowed_ingress_ports also requires a source.

Use Case

Any time allowed_ingress_ports is set to non-default value.

Describe Ideal Solution

Check that a source is set.

Cannot specify multiple security rules of the same type for different porst

Describe the Bug

The cloudposse/mq-broker/aws module is using the cloudposse/mq-broker/aws with version 0.3.1 internally, which fails with below error when passing multiple ports for the same type

# Error
│ Error: [WARN] A duplicate Security Group rule was found on (sg-0d867c5b26555c769). This may be
│ a side effect of a now-fixed Terraform issue causing two security groups with
│ identical attributes but different source_security_group_ids to overwrite each
│ other in the state. See https://github.com/hashicorp/terraform/pull/2376 for more
│ information and instructions for recovery. Error: InvalidPermission.Duplicate: the specified rule "peer: 0.0.0.0/0, ALL, ALLOW" already exists
│       status code: 400, request id: 9573b001-d97f-490d-a18c-0f00c7d8e198
│
│   with module.mq_test.module.security_group.aws_security_group_rule.default["ingress--1-5671-5671-9c87e5e1ed040a443ce1ac8e6d6cf159"],
│   on .terraform/modules/mq_test.security_group/main.tf line 41, in resource "aws_security_group_rule" "default":
│   41: resource "aws_security_group_rule" "default" {

Expected Behavior

Security group should be created with the specified rules

## Steps to Reproduce
Use below root module to deploy the resource passin `security_group_rules` argument
# root module call
module "mq_test" {
  source = "cloudposse/mq-broker/aws"
  version     = "0.15.0"
  [...]
  security_group_rules = [
    {
      type        = "ingress"
      from_port   = 5671
      to_port     = 5671
      protocol    = "-1"
      cidr_blocks = ["0.0.0.0/0"]
      description = "Allow all outbound traffic"
    },
    {
      type        = "ingress"
      from_port   = 8883
      to_port     = 8883
      protocol    = "-1"
      cidr_blocks = ["0.0.0.0/0"]
      description = "Allow all outbound traffic"
    },
    {
      type        = "egress"
      from_port   = 5671
      to_port     = 5671
      protocol    = "-1"
      cidr_blocks = ["0.0.0.0/0"]
      description = "Allow all inbound traffic"
    }
  ]
}

Environment (please complete the following information):

Anything that will help us triage the bug will help. Here are some ideas:

• OS: WSL
• Version: Terraform v1.0.11

• provider registry.terraform.io/hashicorp/aws v3.66.0

Issues with mq broker

aws_mq_broker.default.*.instances.0.endpoints.2 this does not works.

I see errors

• module.amq.output.secondary_stomp_ssl_endpoint: Resource 'aws_mq_broker.default' does not have attribute 'instances.1.endpoints.2' for variable 'aws_mq_broker.default.*.instances.1.endpoints.2'
• module.amq.output.primary_ip_address: Resource 'aws_mq_broker.default' does not have attribute 'instances.0.ip_address' for variable 'aws_mq_broker.default.*.instances.0.ip_address'
• module.amq.output.secondary_ampq_ssl_endpoint: Resource 'aws_mq_broker.default' does not have attribute 'instances.1.endpoints.1' for variable 'aws_mq_broker.default.*.instances.1.endpoints.1'
• module.amq.output.secondary_console_url: Resource 'aws_mq_broker.default' does not have attribute 'instances.1.console_url' for variable 'aws_mq_broker.default.*.instances.1.console_url'
• module.amq.output.primary_mqtt_ssl_endpoint: Resource 'aws_mq_broker.default' does not have attribute 'instances.0.endpoints.3' for variable 'aws_mq_broker.default.*.instances.0.endpoints.3'
• module.amq.output.primary_ssl_endpoint: Resource 'aws_mq_broker.default' does not have attribute 'instances.0.endpoints.0' for variable 'aws_mq_broker.default.*.instances.0.endpoints.0'
• module.amq.output.primary_console_url: Resource 'aws_mq_broker.default' does not have attribute 'instances.0.console_url' for variable 'aws_mq_broker.default.*.instances.0.console_url'
• module.amq.output.secondary_ip_address: Resource 'aws_mq_broker.default' does not have attribute 'instances.1.ip_address' for variable 'aws_mq_broker.default.*.instances.1.ip_address'
• module.amq.output.primary_ampq_ssl_endpoint: Resource 'aws_mq_broker.default' does not have attribute 'instances.0.endpoints.1' for variable 'aws_mq_broker.default.*.instances.0.endpoints.1'
• module.amq.output.primary_wss_endpoint: Resource 'aws_mq_broker.default' does not have attribute 'instances.0.endpoints.4' for variable 'aws_mq_broker.default.*.instances.0.endpoints.4'
• module.amq.output.primary_stomp_ssl_endpoint: Resource 'aws_mq_broker.default' does not have attribute 'instances.0.endpoints.2' for variable 'aws_mq_broker.default.*.instances.0.endpoints.2'
• module.amq.output.secondary_mqtt_ssl_endpoint: Resource 'aws_mq_broker.default' does not have attribute 'instances.1.endpoints.3' for variable 'aws_mq_broker.default.*.instances.1.endpoints.3'
• module.amq.output.secondary_wss_endpoint: Resource 'aws_mq_broker.default' does not have attribute 'instances.1.endpoints.4' for variable 'aws_mq_broker.default.*.instances.1.endpoints.4'
• module.amq.output.secondary_ssl_endpoint: Resource 'aws_mq_broker.default' does not have attribute 'instances.1.endpoints.0' for variable 'aws_mq_broker.default.*.instances.1.endpoints.0'

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

add the ability to specify a configuration

Describe the Feature

The raq aws_mq_broker terraform resource has a configuration block (see official terraform docs). However, this block cannot be specified in this module.

Expected Behavior

The user should be able to specify a aws_mq_configuration terraform resource to the module.

Use Case

The user wants to provide aws_mq_configuration terraform resource to the module.

Describe Ideal Solution

The user wants to provide aws_mq_configuration terraform resource to the module.

Alternatives Considered

N/A

Additional Context

No response

Incorrect protocol for ingress security groups

Describe the Bug

When using the allowed_security_groups or allowed_cidr_blocks inputs they are setting port 0 traffic as allowed. This doesn't work since any broker queue (ActiveMQ or RabbitMQ) isn't on this port. The rule descriptions indicate it should be allowing all traffic so this points to not using the "tcp" protocol.

Expected Behavior

I think the ingress security group rules need to change from protocol = "tcp" to protocol = "-1" like the egress rule does. Even this is a bit wide open IMO when it should be restricting traffic to only the ports needed but at least this will fix it.

Additional Context

• https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule

Excerpt:

Setting protocol = "all" or protocol = -1 with from_port and to_port will result in the EC2 API creating a security group rule with all ports open.

SSM parameters should be optional

Describe the Feature

Hello!

Thank you for this (and other) modules! They are a breeze to use and help us a lot during development.

I would like to propose that SSM parameter creation should be optional. The reason is twofold:

1. always-created SSM parameters is unexpected behavior when compared with rest of your modules (namely: rds, redis)
2. We'd like to manage all SSM parameters in one place (with your ssm module btw), but this module forces us to either duplicate the parameters or split the configuration.

I'm happy to implement this feature if you agree to it and comment with a suggestion for the switch name.

Expected Behavior

SSM parameter creation should be hidden behind an optional flag. Probably set as a default to false?

Use Case

We would like to keep all parameters in one place and set them according to our needs, with values sourced from this module's outputs. That's technically still possible, but the module creates its own parameters, causing a bit of confusion and duplication.

Describe Ideal Solution

SSM parameter creation is hidden behind a flag.
If flag is set to false => no SSM parameter will be created automatically by the module.
If set to true => module works as it does now.

Alternatives Considered

No response

Additional Context

No response

Getting "Audit logging is not supported for RabbitMQ brokers." error when audit logging set to false or not defined

I'm trying to deploy RabbitMQ. It's a test, so the set up is basic and variables are hardcoded.
I'm on Terraform v0.13.0, runnning on macOS.

module "aws_mq_broker"{
    source = "git::https://github.com/cloudposse/terraform-aws-mq-broker.git?ref=0.13.0"
    stage                        = "test"
    name                         = "rabbitmq"
    apply_immediately            = true
    auto_minor_version_upgrade   = false
    deployment_mode              = "CLUSTER_MULTI_AZ"
    engine_type                  = "RabbitMQ"
    engine_version               = "3.9.13"
    host_instance_type           = "mq.m5.large"
    general_log_enabled          = true
    audit_log_enabled            = false
    encryption_enabled           = true
    use_aws_owned_key            = true
    vpc_id                       = "vpc-xxxxx"
    subnet_ids                   = [ "subnet-xxxx", "subnet-xxxx" ]
    security_group_rules        = []
}

terraform plan succeeds, but terraform apply errors out with:

Error: BadRequestException: Audit logging is not supported for RabbitMQ brokers.
{
  RespMetadata: {
    StatusCode: 400,
    RequestID: "5ee2ba47-6465-4acb-9ebd-e3839e8adbe9"
  },
  ErrorAttribute: "logs.audit",
  Message_: "Audit logging is not supported for RabbitMQ brokers."
}

I have tried 1) setting both general and audit logs to false; 2) removing these lines completely. In all scenarios I get the same error. I've seen similar issues discussed on Github and Stackoverflow when using the resource "rabbitmq", but in their case, if I understood correctly, the issue was that they didn't define these variables, so terraform resorted to default, which was set to true.

Invalid count argument mq_application_password

Found a bug? Maybe our Slack Community can help.

Describe the Bug

│ Error: Invalid count argument
│ 
│   on .terraform/modules/main.mq_broker/main.tf line 42, in resource "random_password" "mq_application_password":
│   42:   count   = local.enabled && ! local.mq_application_password_is_set ? 1 : 0
│ 
│ The "count" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created.
│ To work around this, use the -target argument to first apply only the resources that the count depends on.

Expected Behavior

im using the following for random password gen:

resource "random_password" "password" {
  length           = 32
  special          = true
  override_special = "-._~"
}

Environment (please complete the following information):

Anything that will help us triage the bug will help. Here are some ideas:

• OS: OSX, Debian buster
• Version Terraform v1.0.1

The argument "user" was already set at main.tf:104,3-7. Each argument may be set only once.

Error: Attribute redefined

on main.tf line 111, in resource "aws_mq_broker" "default":
111: user = [{

The argument "user" was already set at main.tf:104,3-7. Each argument may be
set only once.

BadRequestException: Specify exactly one user for engine type [RABBITMQ]

When we use this module for creating Rabbitmq, we get an unexpected error in our terraform application:

Error: BadRequestException: Specify exactly one user for engine type [RABBITMQ].
{
RespMetadata: {
StatusCode: 400,
RequestID: "128ea9ae-c2b6-4a3e-a644-2cc3470a2b1a"
},
ErrorAttribute: "users",
Message_: "Specify exactly one user for engine type [RABBITMQ]."
}

Expected Behavior

No bugs, should work fine.

Steps to Reproduce

Run the terraform using this module for RabbitMQ

Provide capability to create configurations for brokers

Describe the Feature

As a user of this module, I need to create configurations with custom values and attach them to the brokers created with this module

Expected Behavior

As a user of this module, I need to create configurations with custom values and attach them to the brokers created with this module

Use Case

We need to update consumer_timeout of our broker

Describe Ideal Solution

provide way to input configuration data

Alternatives Considered

No response

Additional Context

No response

RabbitMQ_Cluster_Mode Unable to create

Unable to Create Cluster mode broker for the rabbitmq engine using this terraform module, seeing SINGLE_INSTANCE and ACTIVE_STANDBY_MULTI_AZ are expected.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

These problems occurred while renovating this repository. View logs.

• WARN: Base branch does not exist - skipping

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• chore(deps): update terraform cloudposse/security-group/aws to v0.4.3 (release/v0)
• chore(deps): update terraform cloudposse/security-group/aws to v2 (release/v0)
• chore(deps): update terraform cloudposse/security-group/aws to v2 (release/v2)

Detected dependencies

Branch main

terraform

sg.tf

• cloudposse/security-group/aws 2.2.0

versions.tf

• aws >= 3.0
• random >= 3.0
• hashicorp/terraform >= 1.0.0

Branch release/v0

terraform

sg.tf

• cloudposse/security-group/aws 0.3.1

versions.tf

• aws >= 2.0
• null >= 2.0
• random >= 2.2.0
• template >= 2.0
• hashicorp/terraform >= 0.13.0

Branch release/v1

terraform

versions.tf

• aws >= 2.0
• null >= 2.0
• random >= 2.2.0
• template >= 2.0
• hashicorp/terraform >= 0.13.0

Branch release/v2

terraform

sg.tf

• cloudposse/security-group/aws 1.0.1

versions.tf

• aws >= 3.0
• random >= 3.0
• hashicorp/terraform >= 0.14.0

---

• Check this box to trigger a request for Renovate to run again on this repository

engine_type seems to support only ActiveMQ

Found a bug? Maybe our Slack Community can help.

Describe the Bug

Documentation states that engine_type can be ActiveMQ or RabbitMQ. Despite that, creation of the resource with engine_type = "RabbitMQ" fails. I can see that at the time of writing (2021-03-08), this module uses aws_mq_broker under the hood, which does not support RabbitMQ yet (see hashicorp/terraform-provider-aws#16108). Probably this is just documentation issue, but if there is a way to spin up aws RabbitMQ broker, please advise. Thanks in advance!

Expected Behavior

Successful creation of the resource

Steps to Reproduce

Steps to reproduce the behavior:

1. Create resource using module:

module "mq_broker" {
  source = "cloudposse/mq-broker/aws"
  namespace                    = "ns"
  stage                        = "test"
  name                         = "rabbit"
  apply_immediately            = true
  auto_minor_version_upgrade   = true
  deployment_mode              = "SINGLE_INSTANCE"
  engine_type                  = "RabbitMQ"
  engine_version               = "3.8.6"
  host_instance_type           = "mq.t3.micro"
  publicly_accessible          = true
  general_log_enabled          = true
  audit_log_enabled            = true
  use_existing_security_groups = true
  existing_security_groups     = [aws_security_group.my.id]
  encryption_enabled           = false
  use_aws_owned_key            = false
  vpc_id                       = data.aws_vpc.my_vpc.id
  subnet_ids                   = data.aws_subnet_ids.my_vpc.ids
  mq_application_user          = data.aws_ssm_parameter.app_login.value
  mq_application_password      = data.aws_ssm_parameter.app_password.value
  mq_admin_user                = data.aws_ssm_parameter.admin_login.value
  mq_admin_password            = data.aws_ssm_parameter.admin_password.value
}

2. Run terraform apply
3. Enter yes
4. See error

Error: expected engine_type to be one of [ACTIVEMQ], got RabbitMQ

  on .terraform/modules/mq_broker/main.tf line 71, in resource "aws_mq_broker" "default":
  71:   engine_type                = var.engine_type

Screenshots

If applicable, add screenshots or logs to help explain your problem.

Environment (please complete the following information):

Anything that will help us triage the bug will help. Here are some ideas:

• OSX 11.2.1
• terraform --version

terraform --version
Terraform v0.14.7
+ provider registry.terraform.io/hashicorp/aws v3.31.0
+ provider registry.terraform.io/hashicorp/null v3.1.0
+ provider registry.terraform.io/hashicorp/random v3.1.0
+ provider registry.terraform.io/hashicorp/template v2.2.0

Additional Context

Error building RabbitMQ : Audit logging is not supported for RabbitMQ brokers

Describe the Bug

When trying to create a RabbitMQ broker using this module, I get an error: Audit logging is not supported for RabbitMQ brokers.

Expected Behavior

Rabbit MQ broker should be created when I set required values.

Example Code

I'm trying to provide most relevant code here. Lots of dependencies I'm leaving off for VPC, but that's all pretty standard.

rabbitmq.tf file

resource "aws_security_group" "rmq_cluster" {
  name        = "${var.appname}-rmq-sg"
  description = "for Rabbit MQ Broker"

  vpc_id = module.vpc.vpc_id

  ingress {
    protocol    = -1
    from_port   = 0
    to_port     = 0
    cidr_blocks = [var.vpccidr]
    description = "Allow all resources in VPC resources to communicate with Rabbit MQ"
  }

  egress {
    protocol    = -1
    from_port   = 0
    to_port     = 0
    cidr_blocks = [var.vpccidr]
    description = "Allow all outbound access from Rabbit MQ"
  }

  tags = local.tags
}

module "mq_broker" {
  source = "cloudposse/mq-broker/aws"
  version = "0.9.0"

  name                         = local.broker_name
  engine_type                  = "RabbitMQ"
  engine_version               = var.rmq_engine_version
  host_instance_type           = var.rmq_host_instance_type
  deployment_mode              = var.rmq_deployment_mode
  vpc_id                       = module.vpc.vpc_id
  subnet_ids                   = local.rmq_subnet_list
  use_existing_security_groups = true
  audit_log_enabled            = false
  existing_security_groups     = [aws_security_group.rmq_cluster.id]
  mq_application_user          = superuser
  mq_application_password      = unbreakablepassword
  tags                         = local.tags
}

vars.tfvars file

rmq_host_instance_type    = "mq.t3.micro"
rmq_engine_version        = "3.8.6"
rmq_deployment_mode       = "SINGLE_INSTANCE"

locals.tf file

broker_name      = "${var.appname}-${var.env}-rmq-${random_string.suffix.result}"
rmq_subnet_list = var.rmq_deployment_mode == "SINGLE_INSTANCE" ? [module.vpc.private_subnets.0] : module.vpc.private_subnets

Error Output

Error: BadRequestException: Audit logging is not supported for RabbitMQ brokers.
{
  RespMetadata: {
    StatusCode: 400,
    RequestID: "2564760b-18ab-4e41-9179-8f6e437e0dbe"
  },
  ErrorAttribute: "logs.audit",
  Message_: "Audit logging is not supported for RabbitMQ brokers."
}

  on .terraform/modules/mq_broker/main.tf line 73, in resource "aws_mq_broker" "default":
  73: resource "aws_mq_broker" "default" {