cloudposse / terraform-aws-mq-broker Goto Github PK
View Code? Open in Web Editor NEWTerraform module for provisioning an AmazonMQ broker
Home Page: https://cloudposse.com/accelerate
License: Apache License 2.0
Terraform module for provisioning an AmazonMQ broker
Home Page: https://cloudposse.com/accelerate
License: Apache License 2.0
╷
│ Error: updating MQ Broker (b-f9764a70-79ab-41de-ba60-2f722296ba50) security groups: BadRequestException: Changing security groups is not supported for RabbitMQ brokers.
│ {
│ RespMetadata: {
│ StatusCode: 400,
│ RequestID: "52d88252-71f0-4800-9c41-aa8455b174bf"
│ },
│ ErrorAttribute: "securityGroups",
│ Message_: "Changing security groups is not supported for RabbitMQ brokers."
│ }
│
│ with module.rabbitmq.aws_mq_broker.default[0],
│ on ../../modules/mq/main.tf line 89, in resource "aws_mq_broker" "default":
│ 89: resource "aws_mq_broker" "default" {
│
╵
id for security group don't change, or recreate rabbitMQ broker whe security_id was change
allowed_security_group_ids
No response
No response
I'm calling terraform-aws-mq-broker v3.1.0 from terragrunt and get the following error on a plan. It looks like the module.this.id
has no value and fails the two tests. I read that these modules should be compatible with the terragrunt.
The main.tf line 91 seems to be the issue.
broker_name = module.this.id
I'm not sure why the error is being thrown and assume it's related to the fact that the id is only available after the module is applied? What's the fix or what am I doing wrong in trying to use cp modules with terragrunt?
│ Error: expected length of broker_name to be in the range (1 - 50), got
│
│ with module.mq_broker.aws_mq_broker.default[0],
│ on .terraform/modules/mq_broker/main.tf line 91, in resource "aws_mq_broker" "default":
│ 91: broker_name = module.this.id
│
╵
╷
│ Error: expected value of broker_name to match regular expression "^[0-9A-Za-z_-]+$", got
│
│ with module.mq_broker.aws_mq_broker.default[0],
│ on .terraform/modules/mq_broker/main.tf line 91, in resource "aws_mq_broker" "default":
│ 91: broker_name = module.this.id
│
╵
Releasing state lock. This may take a few moments...
ERRO[0055] Terraform invocation failed in /app/infrastructure-live/dev/ca-central-1/dev/messaging/rabbitmq/.terragrunt-cache/DzpSoeycWeukzz45-SRvPTXf7Qw/aYg6jvKMeoPdWBiN8E8O6YgHPKw prefix=[dev/ca-central-1/dev/messaging/rabbitmq]
ERRO[0055] 1 error occurred:
* [/app/infrastructure-live/dev/ca-central-1/dev/messaging/rabbitmq/.terragrunt-cache/DzpSoeycWeukzz45-SRvPTXf7Qw/aYg6jvKMeoPdWBiN8E8O6YgHPKw] exit status 1
Module applies without error
Use the module with terragrunt version v0.48.3 based on the examples/complete.
No response
terragrunt version v0.48.3
Terraform v1.5.4
on linux_amd64
No response
When trying the RabbitMQ option the code fails,
│ Error: expected engine_type to be one of [ACTIVEMQ], got RabbitMQ
│
│ with module.mq_broker.aws_mq_broker.default[0],
│ on .terraform/modules/mq_broker/main.tf line 91, in resource "aws_mq_broker" "default":
│ 91: engine_type = var.engine_type
│
╵
using engine_type=RabbitMQ
The overwrite
argument on aws_ssm_parameter
has been deprecated:
│ Warning: Argument is deprecated
│
│ with module.mq.module.mq_broker.aws_ssm_parameter.mq_application_username[0],
│ on .terraform/modules/mq.mq_broker/main.tf line 74, in resource "aws_ssm_parameter" "mq_application_username":
│ 74: overwrite = var.overwrite_ssm_parameter
│
│ this attribute has been deprecated
See docs for reference: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter#overwrite
No deprecation warnings are printed.
Use terraform-aws-mq-broker 3.1.0 with the aws provider 5.x.
No response
No response
No response
In the documentation, it says we can use enabled to disable/enable the resource creation, however, it has not been fully implemented. Can someone please look into it?
Additional feedback when allowed_ingress_ports
argument is set, but no source SG or CIDR is set.
Error or Warning output that setting allowed_ingress_ports
also requires a source.
Any time allowed_ingress_ports
is set to non-default value.
Check that a source is set.
The cloudposse/mq-broker/aws
module is using the cloudposse/mq-broker/aws
with version 0.3.1
internally, which fails with below error when passing multiple ports for the same type
# Error
│ Error: [WARN] A duplicate Security Group rule was found on (sg-0d867c5b26555c769). This may be
│ a side effect of a now-fixed Terraform issue causing two security groups with
│ identical attributes but different source_security_group_ids to overwrite each
│ other in the state. See https://github.com/hashicorp/terraform/pull/2376 for more
│ information and instructions for recovery. Error: InvalidPermission.Duplicate: the specified rule "peer: 0.0.0.0/0, ALL, ALLOW" already exists
│ status code: 400, request id: 9573b001-d97f-490d-a18c-0f00c7d8e198
│
│ with module.mq_test.module.security_group.aws_security_group_rule.default["ingress--1-5671-5671-9c87e5e1ed040a443ce1ac8e6d6cf159"],
│ on .terraform/modules/mq_test.security_group/main.tf line 41, in resource "aws_security_group_rule" "default":
│ 41: resource "aws_security_group_rule" "default" {
Security group should be created with the specified rules
## Steps to Reproduce
Use below root module to deploy the resource passin `security_group_rules` argument
# root module call
module "mq_test" {
source = "cloudposse/mq-broker/aws"
version = "0.15.0"
[...]
security_group_rules = [
{
type = "ingress"
from_port = 5671
to_port = 5671
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
description = "Allow all outbound traffic"
},
{
type = "ingress"
from_port = 8883
to_port = 8883
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
description = "Allow all outbound traffic"
},
{
type = "egress"
from_port = 5671
to_port = 5671
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
description = "Allow all inbound traffic"
}
]
}
Anything that will help us triage the bug will help. Here are some ideas:
aws_mq_broker.default.*.instances.0.endpoints.2
this does not works.
I see errors
There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.
Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.
The raq aws_mq_broker
terraform resource has a configuration
block (see official terraform docs). However, this block cannot be specified in this module.
The user should be able to specify a aws_mq_configuration
terraform resource to the module.
The user wants to provide aws_mq_configuration
terraform resource to the module.
The user wants to provide aws_mq_configuration
terraform resource to the module.
N/A
No response
When using the allowed_security_groups
or allowed_cidr_blocks
inputs they are setting port 0
traffic as allowed. This doesn't work since any broker queue (ActiveMQ or RabbitMQ) isn't on this port. The rule descriptions indicate it should be allowing all traffic so this points to not using the "tcp" protocol.
I think the ingress security group rules need to change from protocol = "tcp"
to protocol = "-1"
like the egress rule does. Even this is a bit wide open IMO when it should be restricting traffic to only the ports needed but at least this will fix it.
Excerpt:
Setting protocol = "all" or protocol = -1 with from_port and to_port will result in the EC2 API creating a security group rule with all ports open.
Hello!
Thank you for this (and other) modules! They are a breeze to use and help us a lot during development.
I would like to propose that SSM parameter creation should be optional. The reason is twofold:
I'm happy to implement this feature if you agree to it and comment with a suggestion for the switch name.
SSM parameter creation should be hidden behind an optional flag. Probably set as a default to false?
We would like to keep all parameters in one place and set them according to our needs, with values sourced from this module's outputs. That's technically still possible, but the module creates its own parameters, causing a bit of confusion and duplication.
SSM parameter creation is hidden behind a flag.
If flag is set to false => no SSM parameter will be created automatically by the module.
If set to true => module works as it does now.
No response
No response
I'm trying to deploy RabbitMQ. It's a test, so the set up is basic and variables are hardcoded.
I'm on Terraform v0.13.0, runnning on macOS.
module "aws_mq_broker"{
source = "git::https://github.com/cloudposse/terraform-aws-mq-broker.git?ref=0.13.0"
stage = "test"
name = "rabbitmq"
apply_immediately = true
auto_minor_version_upgrade = false
deployment_mode = "CLUSTER_MULTI_AZ"
engine_type = "RabbitMQ"
engine_version = "3.9.13"
host_instance_type = "mq.m5.large"
general_log_enabled = true
audit_log_enabled = false
encryption_enabled = true
use_aws_owned_key = true
vpc_id = "vpc-xxxxx"
subnet_ids = [ "subnet-xxxx", "subnet-xxxx" ]
security_group_rules = []
}
terraform plan
succeeds, but terraform apply
errors out with:
Error: BadRequestException: Audit logging is not supported for RabbitMQ brokers.
{
RespMetadata: {
StatusCode: 400,
RequestID: "5ee2ba47-6465-4acb-9ebd-e3839e8adbe9"
},
ErrorAttribute: "logs.audit",
Message_: "Audit logging is not supported for RabbitMQ brokers."
}
I have tried 1) setting both general and audit logs to false; 2) removing these lines completely. In all scenarios I get the same error. I've seen similar issues discussed on Github and Stackoverflow when using the resource "rabbitmq", but in their case, if I understood correctly, the issue was that they didn't define these variables, so terraform resorted to default, which was set to true.
Found a bug? Maybe our Slack Community can help.
│ Error: Invalid count argument
│
│ on .terraform/modules/main.mq_broker/main.tf line 42, in resource "random_password" "mq_application_password":
│ 42: count = local.enabled && ! local.mq_application_password_is_set ? 1 : 0
│
│ The "count" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created.
│ To work around this, use the -target argument to first apply only the resources that the count depends on.
im using the following for random password gen:
resource "random_password" "password" {
length = 32
special = true
override_special = "-._~"
}
Anything that will help us triage the bug will help. Here are some ideas:
Error: Attribute redefined
on main.tf line 111, in resource "aws_mq_broker" "default":
111: user = [{
The argument "user" was already set at main.tf:104,3-7. Each argument may be
set only once.
When we use this module for creating Rabbitmq, we get an unexpected error in our terraform application:
Error: BadRequestException: Specify exactly one user for engine type [RABBITMQ].
{
RespMetadata: {
StatusCode: 400,
RequestID: "128ea9ae-c2b6-4a3e-a644-2cc3470a2b1a"
},
ErrorAttribute: "users",
Message_: "Specify exactly one user for engine type [RABBITMQ]."
}
No bugs, should work fine.
Run the terraform using this module for RabbitMQ
As a user of this module, I need to create configurations with custom values and attach them to the brokers created with this module
As a user of this module, I need to create configurations with custom values and attach them to the brokers created with this module
We need to update consumer_timeout of our broker
provide way to input configuration data
No response
No response
Unable to Create Cluster mode broker for the rabbitmq engine using this terraform module, seeing SINGLE_INSTANCE and ACTIVE_STANDBY_MULTI_AZ are expected.
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
These problems occurred while renovating this repository. View logs.
These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.
terraform
sg.tf
cloudposse/security-group/aws 2.2.0
versions.tf
aws >= 3.0
random >= 3.0
hashicorp/terraform >= 1.0.0
terraform
sg.tf
cloudposse/security-group/aws 0.3.1
versions.tf
aws >= 2.0
null >= 2.0
random >= 2.2.0
template >= 2.0
hashicorp/terraform >= 0.13.0
terraform
versions.tf
aws >= 2.0
null >= 2.0
random >= 2.2.0
template >= 2.0
hashicorp/terraform >= 0.13.0
terraform
sg.tf
cloudposse/security-group/aws 1.0.1
versions.tf
aws >= 3.0
random >= 3.0
hashicorp/terraform >= 0.14.0
Found a bug? Maybe our Slack Community can help.
Documentation states that engine_type
can be ActiveMQ or RabbitMQ. Despite that, creation of the resource with engine_type = "RabbitMQ"
fails. I can see that at the time of writing (2021-03-08), this module uses aws_mq_broker
under the hood, which does not support RabbitMQ yet (see hashicorp/terraform-provider-aws#16108). Probably this is just documentation issue, but if there is a way to spin up aws RabbitMQ broker, please advise. Thanks in advance!
Successful creation of the resource
Steps to reproduce the behavior:
module "mq_broker" {
source = "cloudposse/mq-broker/aws"
namespace = "ns"
stage = "test"
name = "rabbit"
apply_immediately = true
auto_minor_version_upgrade = true
deployment_mode = "SINGLE_INSTANCE"
engine_type = "RabbitMQ"
engine_version = "3.8.6"
host_instance_type = "mq.t3.micro"
publicly_accessible = true
general_log_enabled = true
audit_log_enabled = true
use_existing_security_groups = true
existing_security_groups = [aws_security_group.my.id]
encryption_enabled = false
use_aws_owned_key = false
vpc_id = data.aws_vpc.my_vpc.id
subnet_ids = data.aws_subnet_ids.my_vpc.ids
mq_application_user = data.aws_ssm_parameter.app_login.value
mq_application_password = data.aws_ssm_parameter.app_password.value
mq_admin_user = data.aws_ssm_parameter.admin_login.value
mq_admin_password = data.aws_ssm_parameter.admin_password.value
}
terraform apply
yes
Error: expected engine_type to be one of [ACTIVEMQ], got RabbitMQ
on .terraform/modules/mq_broker/main.tf line 71, in resource "aws_mq_broker" "default":
71: engine_type = var.engine_type
If applicable, add screenshots or logs to help explain your problem.
Anything that will help us triage the bug will help. Here are some ideas:
terraform --version
terraform --version
Terraform v0.14.7
+ provider registry.terraform.io/hashicorp/aws v3.31.0
+ provider registry.terraform.io/hashicorp/null v3.1.0
+ provider registry.terraform.io/hashicorp/random v3.1.0
+ provider registry.terraform.io/hashicorp/template v2.2.0
When trying to create a RabbitMQ broker using this module, I get an error: Audit logging is not supported for RabbitMQ brokers.
Rabbit MQ broker should be created when I set required values.
I'm trying to provide most relevant code here. Lots of dependencies I'm leaving off for VPC, but that's all pretty standard.
rabbitmq.tf file
resource "aws_security_group" "rmq_cluster" {
name = "${var.appname}-rmq-sg"
description = "for Rabbit MQ Broker"
vpc_id = module.vpc.vpc_id
ingress {
protocol = -1
from_port = 0
to_port = 0
cidr_blocks = [var.vpccidr]
description = "Allow all resources in VPC resources to communicate with Rabbit MQ"
}
egress {
protocol = -1
from_port = 0
to_port = 0
cidr_blocks = [var.vpccidr]
description = "Allow all outbound access from Rabbit MQ"
}
tags = local.tags
}
module "mq_broker" {
source = "cloudposse/mq-broker/aws"
version = "0.9.0"
name = local.broker_name
engine_type = "RabbitMQ"
engine_version = var.rmq_engine_version
host_instance_type = var.rmq_host_instance_type
deployment_mode = var.rmq_deployment_mode
vpc_id = module.vpc.vpc_id
subnet_ids = local.rmq_subnet_list
use_existing_security_groups = true
audit_log_enabled = false
existing_security_groups = [aws_security_group.rmq_cluster.id]
mq_application_user = superuser
mq_application_password = unbreakablepassword
tags = local.tags
}
vars.tfvars file
rmq_host_instance_type = "mq.t3.micro"
rmq_engine_version = "3.8.6"
rmq_deployment_mode = "SINGLE_INSTANCE"
locals.tf file
broker_name = "${var.appname}-${var.env}-rmq-${random_string.suffix.result}"
rmq_subnet_list = var.rmq_deployment_mode == "SINGLE_INSTANCE" ? [module.vpc.private_subnets.0] : module.vpc.private_subnets
Error: BadRequestException: Audit logging is not supported for RabbitMQ brokers.
{
RespMetadata: {
StatusCode: 400,
RequestID: "2564760b-18ab-4e41-9179-8f6e437e0dbe"
},
ErrorAttribute: "logs.audit",
Message_: "Audit logging is not supported for RabbitMQ brokers."
}
on .terraform/modules/mq_broker/main.tf line 73, in resource "aws_mq_broker" "default":
73: resource "aws_mq_broker" "default" {
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.