terraform-ibm-modules/terraform-ibm-event-streams

Module that supports provisioning and configuring IBM Cloud Event Streams

terraform-ibm-event-streams's Issues

Review Event Streams DA diagram

Review the diagram(s) in the reference-architectures directory...

• Do they contain accurate relevant information
• Are they using approved icons / objects?

[terraform-ibm-event-streams] Change the name of the DA flavor / variation

We don't want to use the name fscloud as the flavor / variation for the DA. Actions:

• Rename dir: solutions/fscloud -> solutions/standard
• ibm_catalog.json updates:
  • Update working_directory to new dir (solutions/standard)
  • Update flavor name and label to standard / Standard
• Update variation name in .catalog-onboard-pipeline.yaml

FSCloud profile for EventStreams

• Create FSCloud profile submodule
• Create FSCloud example that calls the submodule
• Ensure CRA is pointing to new FSCloud example
• Ensure SCC scan is passing on the EventStreams instance created by the FSCloud profile

Review all Event Streams DA related documentation

The Event Streams DA has 1 flavor currently:

• fscloud (probably need to change this flavor name?)

The following items will need to be reviewed and updated where required:

• DA variable descriptions (in the DA's variables.tf file)
• The information in the ibm_catalog.json (labels, descriptions, features etc).

Update the provider version to use minimum 1.65.0

In the CBR module, the provider constraints are updated to >= 1.65.0, hence the need to update.

Add back KMS auth policy support to Event Streams module

As per this comment, the backend limitations that forced us to remove auth policy support in v2.0.0 can now be added back in.
Docs should be updated, and we should probably update the v2.0.0 release notes to mention that we added the support back in a newer version (version tbc after PR is merged)

Release Pipeline failed

For details see: https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/actions/runs/8050884839

Test failure due to auth policy not being created

FSCloud tests were failing due to auth policy not being created before the es instance.

https://cloud.ibm.com/devops/pipelines/tekton/3ed3b04a-8f9f-4db1-ac37-8d6dfb7951e9/runs/2db6a9b3-c316-4e5b-8a25-8e9021bb14d1/run-tests/run-tests?env_id=ibm:yp:us-south

TestRunFSCloudExample 2023-11-16T06:10:16Z logger.go:66: │     "Result": {
TestRunFSCloudExample 2023-11-16T06:10:16Z logger.go:66: │         "details": "{\"description\":\"Please ensure the KMS instance: crn:v1:bluemix:public:hs-crypto:us-south:a/123 exists and Event Streams service has been authorized to access the selected KMS instance\",\"error\":\"Bad Request\",\"incident_id\":\"bss-34443\"}\n",
TestRunFSCloudExample 2023-11-16T06:10:16Z logger.go:66: │         "error_code": "RC-ServiceBrokerErrorResponse",
TestRunFSCloudExample 2023-11-16T06:10:16Z logger.go:66: │         "message": "Please contact the Service Provider for this error. [400, Bad Request] Please ensure the KMS instance: crn:v1:bluemix:public:hs-crypto:us-south:a/123:key:123 exists and Event Streams service has been authorized to access the selected KMS instance",
TestRunFSCloudExample 2023-11-16T06:10:16Z logger.go:66: │         "status_code": 422,
TestRunFSCloudExample 2023-11-16T06:10:16Z logger.go:66: │         "transaction_id": "bss-34443"
TestRunFSCloudExample 2023-11-16T06:10:16Z logger.go:66: │     },

[terraform-ibm-event-streams] Add CBR support

Add support to this module to create CBRs (Context Based Restrictions) for finer grained network filtering. See terraform-ibm-icd-mongodb or terraform-ibm-icd-postgresql for implementation and be consistent.

Event Streams DA needs some supporting docs for complex object type inputs

The schemas, topics and cbr_rules variables are complex object types, which users will find hard to figure out the required format to be entered in Projects UI.
Until Projects support some kind of tool tip (tracked internally here), the best thing we can do is add a supporting doc which the variable descriptions can link to.
We did this for KMS DA -> https://github.com/terraform-ibm-modules/terraform-ibm-kms-all-inclusive/blob/main/solutions/standard/DA-keys.md

Add support to Event Stream DA to create KMS key

Add support to Event Stream DA to create KMS key. Considerations:

• Support creating KMS key in existing KMS instance
• Support the use case where KMS might be in a different account:
  • Add new optional variable ibmcloud_kms_api_key. This should be used in a new provider block like so:

    provider "ibm" {
      alias            = "kms"
      ibmcloud_api_key = var.ibmcloud_kms_api_key != null ? var.ibmcloud_kms_api_key : var.ibmcloud_api_key
      region           = local.kms_region # this value should be parsed from the existing KMS CRN 
    }
    
    provider "ibm" {
      ibmcloud_api_key = var.ibmcloud_api_key
    }
    

  • The kms module block used to create the KMS key should use the kms provider alias (see above).
• Review all of the variable descriptions and readme markdowns to ensure its clear that it supports KMS in a different account using the ibmcloud_kms_api_key variable.

Region variable is missing from Event Streams fscloud submodule

There is no region variable in https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/blob/main/modules/fscloud/variables.tf meaning it will always deploy to the default region thats set in the root level module (which is us-south).

Actions:

• Expose region variable in fscloud submodule
• Update the DA to pass its region variable into the fscloud submodule call (NOTE: It has a region variable but its only currently used in the provider config)