terraform-ibm-modules/terraform-ibm-cos

Configures an IBM Cloud Object Storage instance and bucket

Cloud Object Storage module

Use this module to provision and configure an IBM Cloud Object Storage instance and bucket.

In addition, a buckets submodule supports creating multiple buckets in an existing instance.

You can configure the following aspects of your instances:

• Key management service (KMS) encryption
• Activity tracking and auditing
• Monitoring
• Data retention, lifecycle and archiving options

Overview

• terraform-ibm-cos
• Submodules
  • buckets
  • fscloud
• Examples
  • Advanced example
  • Basic example
  • Bucket replication example
  • Financial Services compliant example
  • One Rate plan example
• Contributing

terraform-ibm-cos

Usage

provider "ibm" {
  ibmcloud_api_key = "XXXXXXXXXX"
  region           = "us-south"
}

# Creates:
# - COS instance
# - COS buckets with retention, encryption, monitoring and activity tracking
module "cos_module" {
  source                     = "terraform-ibm-modules/cos/ibm"
  version                    = "X.X.X" # Replace "X.X.X" with a release version to lock into a specific release
  resource_group_id          = "xxXXxxXXxXxXXXXxxXxxxXXXXxXXXXX"
  region                     = "us-south"
  cos_instance_name          = "my-cos-instance"
  bucket_name                = "my-cos-bucket"
  existing_kms_instance_guid = "xxxxxxxx-XXXX-XXXX-XXXX-xxxxxxxx"
  kms_key_crn                = "crn:v1:bluemix:public:kms:us-south:a/xxXXxxXXxXxXXXXxxXxxxXXXXxXXXXX:xxxxxx-XXXX-XXXX-XXXX-xxxxxx:key:xxxxxx-XXXX-XXXX-XXXX-xxxxxx"
  sysdig_crn                 = "crn:v1:bluemix:public:sysdig-monitor:us-south:a/xxXXxxXXxXxXXXXxxXxxxXXXXxXXXXX:xxXXxxXXxXxXXXXxxXxxxXXXXxXXXXX::"
  activity_tracker_crn       = "crn:v1:bluemix:public:logdnaat:us-south:a/xxXXxxXXxXxXXXXxxXxxxXXXXxXXXXX:xxXXxxXXxXxXXXXxxXxxxXXXXxXXXXX::"
}

# Creates additional buckets in existing instance:
module "additional_cos_bucket" {
  source                   = "terraform-ibm-modules/cos/ibm"
  version                  = "X.X.X" # Replace "X.X.X" with a release version to lock into a specific release
  region                   = "us-south"
  create_cos_instance      = false
  sysdig_crn               = "crn:v1:bluemix:public:sysdig-monitor:us-south:a/xxXXxxXXxXxXXXXxxXxxxXXXXxXXXXX:xxXXxxXXxXxXXXXxxXxxxXXXXxXXXXX::"
  activity_tracker_crn     = "crn:v1:bluemix:public:logdnaat:us-south:a/xxXXxxXXxXxXXXXxxXxxxXXXXxXXXXX:xxXXxxXXxXxXXXXxxXxxxXXXXxXXXXX::"
  existing_cos_instance_id = module.cos_module.cos_instance_id
  kms_key_crn              = "crn:v1:bluemix:public:kms:us-south:a/xxXXxxXXxXxXXXXxxXxxxXXXXxXXXXX:xxxxxx-XXXX-XXXX-XXXX-xxxxxx:key:xxxxxx-XXXX-XXXX-XXXX-xxxxxx"
}

# Creates additional Cloud Object Storage buckets using the buckets sub module
module "cos_buckets" {
  source  = "terraform-ibm-modules/cos/ibm//modules/buckets"
  version = "X.X.X" # Replace "X.X.X" with a release version to lock into a specific release
  bucket_configs = [
    {
      bucket_name          = "my-encrypted-bucket"
      kms_key_crn          = "crn:v1:bluemix:public:kms:us-south:a/xxXXxxXXxXxXXXXxxXxxxXXXXxXXXXX:xxxxxx-XXXX-XXXX-XXXX-xxxxxx:key:xxxxxx-XXXX-XXXX-XXXX-xxxxxx"
      region_location      = "us-south"
      resource_instance_id = module.cos_module.cos_instance_id
    },
    {
      bucket_name            = "my-versioned-bucket"
      kms_encryption_enabled = false
      region_location        = "us-south"
      resource_instance_id   = module.cos_module.cos_instance_id
      object_versioning = {
        enable = true
      }
    },
    {
      bucket_name            = "my-archive-bucket"
      kms_encryption_enabled = false
      region_location        = "us-south"
      resource_instance_id   = module.cos_module.cos_instance_id
      archive_rule = {
        days   = 90
        enable = true
        type   = "Accelerated"
      }
      expire_rule = {
        days   = 90
        enable = true
      }
    }
  ]
}

Required IAM access policies

You need the following permissions to run this module.

• Account Management
  • Resource Group service
    • Viewer platform access
• IAM Services
  • IBM Cloud Activity Tracker service
    • Editor platform access
    • Manager service access
  • IBM Cloud Monitoring service
    • Editor platform access
    • Manager service access
  • IBM Cloud Object Storage service
    • Editor platform access
    • Manager service access

Requirements

Name	Version
terraform	>= 1.4.0, <1.7.0
ibm	>= 1.62.0, < 2.0.0
random	>= 3.5.1, < 4.0.0
time	>= 0.9.1, < 1.0.0

Modules

Name	Source	Version
bucket_cbr_rule	terraform-ibm-modules/cbr/ibm//modules/cbr-rule-module	1.20.1
instance_cbr_rule	terraform-ibm-modules/cbr/ibm//modules/cbr-rule-module	1.20.1

Resources

Name	Type
ibm_cos_bucket.cos_bucket	resource
ibm_cos_bucket.cos_bucket1	resource
ibm_iam_authorization_policy.policy	resource
ibm_resource_instance.cos_instance	resource
ibm_resource_key.resource_keys	resource
ibm_resource_tag.cos_access_tag	resource
random_string.bucket_name_suffix	resource
time_sleep.wait_for_authorization_policy	resource

Inputs

Name	Description	Type	Default	Required
access_tags	A list of access tags to apply to the cos instance created by the module, see https://cloud.ibm.com/docs/account?topic=account-access-tags-tutorial for more details	list(string)	[]	no
activity_tracker_crn	Activity tracker crn for COS bucket (Optional)	string	null	no
add_bucket_name_suffix	Add random generated suffix (4 characters long) to the newly provisioned COS bucket name (Optional).	bool	false	no
archive_days	Specifies the number of days when the archive rule action takes effect. Only used if 'create_cos_bucket' is true. This must be set to null when when using var.cross_region_location as archive data is not supported with this feature.	number	90	no
archive_type	Specifies the storage class or archive type to which you want the object to transition. Only used if 'create_cos_bucket' is true.	string	"Glacier"	no
bucket_cbr_rules	(Optional, list) List of CBR rules to create for the bucket	list(object({ description = string account_id = string rule_contexts = list(object({ attributes = optional(list(object({ name = string value = string }))) })) enforcement_mode = string tags = optional(list(object({ name = string value = string })), []) operations = optional(list(object({ api_types = list(object({ api_type_id = string })) }))) }))	[]	no
bucket_name	The name to give the newly provisioned COS bucket. Only required if 'create_cos_bucket' is true.	string	null	no
bucket_storage_class	the storage class of the newly provisioned COS bucket. Only required if 'create_cos_bucket' is true. Supported values are 'standard', 'vault', 'cold', 'smart' and onerate_active.	string	"standard"	no
cos_instance_name	The name to give the cloud object storage instance that will be provisioned by this module. Only required if 'create_cos_instance' is true.	string	null	no
cos_location	Location to provision the cloud object storage instance. Only used if 'create_cos_instance' is true.	string	"global"	no
cos_plan	Plan to be used for creating cloud object storage instance. Only used if 'create_cos_instance' it true.	string	"standard"	no
cos_tags	Optional list of tags to be added to cloud object storage instance. Only used if 'create_cos_instance' it true.	list(string)	[]	no
create_cos_bucket	Set as true to create a new Cloud Object Storage bucket	bool	true	no
create_cos_instance	Set as true to create a new Cloud Object Storage instance.	bool	true	no
cross_region_location	Specify the cross-regional bucket location. Supported values are 'us', 'eu', and 'ap'. If you pass a value for this, ensure to set the value of var.region and var.single_site_location to null.	string	null	no
existing_cos_instance_id	The ID of an existing cloud object storage instance. Required if 'var.create_cos_instance' is false.	string	null	no
existing_kms_instance_guid	The GUID of the Key Protect or Hyper Protect instance in which the key specified in var.kms_key_crn is coming from. Required if var.skip_iam_authorization_policy is false in order to create an IAM Access Policy to allow Key Protect or Hyper Protect to access the newly created COS instance.	string	null	no
expire_days	Specifies the number of days when the expire rule action takes effect. Only used if 'create_cos_bucket' is true.	number	365	no
force_delete	Deletes all the objects in the COS Bucket before bucket is deleted.	bool	true	no
hard_quota	Sets a maximum amount of storage (in bytes) available for a bucket. If it is set to null then quota is disabled.	number	null	no
instance_cbr_rules	(Optional, list) List of CBR rules to create for the instance	list(object({ description = string account_id = string rule_contexts = list(object({ attributes = optional(list(object({ name = string value = string }))) })) enforcement_mode = string tags = optional(list(object({ name = string value = string })), []) operations = optional(list(object({ api_types = list(object({ api_type_id = string })) }))) }))	[]	no
kms_encryption_enabled	Set as true to use KMS key encryption to encrypt data in COS bucket (only applicable when var.create_cos_bucket is true).	bool	true	no
kms_key_crn	CRN of the KMS key to use to encrypt the data in the COS bucket. Required if var.encryption_enabled and var.create_cos_bucket are true.	string	null	no
management_endpoint_type_for_bucket	The type of endpoint for the IBM terraform provider to use to manage the bucket. (public, private or direct)	string	"public"	no
object_versioning_enabled	Enable object versioning to keep multiple versions of an object in a bucket. Cannot be used with retention rule. Only used if 'create_cos_bucket' is true.	bool	false	no
region	The region to provision the bucket. If you pass a value for this, do not pass one for var.cross_region_location or var.single_site_location.	string	"us-south"	no
resource_group_id	The resource group ID where The COS instance will be provisioned. It is required if setting input variable create_cos_instance to true.	string	null	no
resource_keys	The definition of any resource keys to be generated	list(object({ name = string generate_hmac_credentials = optional(bool, false) role = optional(string, "Reader") service_id_crn = optional(string, null) }))	[]	no
retention_default	Specifies default duration of time an object that can be kept unmodified for COS bucket. Only used if 'create_cos_bucket' is true.	number	90	no
retention_enabled	Retention enabled for COS bucket. Only used if 'create_cos_bucket' is true.	bool	false	no
retention_maximum	Specifies maximum duration of time an object that can be kept unmodified for COS bucket. Only used if 'create_cos_bucket' is true.	number	350	no
retention_minimum	Specifies minimum duration of time an object must be kept unmodified for COS bucket. Only used if 'create_cos_bucket' is true.	number	90	no
retention_permanent	Specifies a permanent retention status either enable or disable for COS bucket. Only used if 'create_cos_bucket' is true.	bool	false	no
single_site_location	Specify the single site bucket location. If you pass a value for this, ensure to set the value of var.region and var.cross_region_location to null.	string	null	no
skip_iam_authorization_policy	Set to true to skip the creation of an IAM authorization policy that permits the COS instance created to read the encryption key from the KMS instance in existing_kms_instance_guid. WARNING: An authorization policy must exist before an encrypted bucket can be created	bool	false	no
sysdig_crn	Sysdig Monitoring crn for COS bucket (Optional)	string	null	no

Outputs

Name	Description
bucket_cbr_rules	COS bucket rules
bucket_crn	Bucket CRN
bucket_id	Bucket id
bucket_name	Bucket name
bucket_region	Bucket region if you create a regional bucket
bucket_storage_class	Bucket Storage Class
cbr_rule_ids	List of all rule ids
cos_instance_crn	The CRN of the Cloud Object Storage instance
cos_instance_guid	The GUID of the Cloud Object Storage instance
cos_instance_id	The ID of the Cloud Object Storage instance
cos_instance_name	The name of the Cloud Object Storage instance
instance_cbr_rules	COS instance rules
kms_key_crn	The CRN of the KMS key used to encrypt the COS bucket
resource_group_id	Resource Group ID
resource_keys	List of resource keys
s3_endpoint_direct	S3 direct endpoint
s3_endpoint_private	S3 private endpoint
s3_endpoint_public	S3 public endpoint

Contributing

You can report issues and request features for this module in GitHub issues in the module repo. See Report an issue or request a feature.

To set up your local development environment, see Local development setup in the project documentation.

terraform-ibm-cos's Issues

Complete example does not execute when region is set to eu-gb

╷
│ Error: InvalidRequest: The region provided in the KMS key CRN does not provide cross region support.
│ status code: 400, request id: a6d17a58-f715-479e-bb0c-1fe53a149f00, host id:
│
│ with module.cos_bucket2.ibm_cos_bucket.cos_bucket[0],
│ on ../../main.tf line 77, in resource "ibm_cos_bucket" "cos_bucket":
│ 77: resource "ibm_cos_bucket" "cos_bucket" {
│

Add option to tie resource_key to an existing serviceId

Description

What:

• Current logic always creates a new ServiceId at https://github.com/terraform-ibm-modules/terraform-ibm-cos/blob/main/main.tf#L49
• There is an option to pass a reference to an existing ServiceId
• Module should take (optionally) the serviceId reference as input

'encyption_enabled' field name is misleading

Minor issue but could lead to confusion.

encryption_enabled controls whether kms encryption is enable (key protect / hpcs). If set to false, cos bucket is still encrypted but with ibm managed keys.

Variable descriptions in COS fscloud submodule need to be updated

The descriptions of the below variable in the FSCloud submodule seem to indicated they are required if 'create_cos_bucket' is true, however there is no create_cos_bucket variable in the FSCloud submodule. They are also not required variables at all (which I think should be correct - the user can optionally opt in).

variable "activity_tracker_crn" {
  type        = string
  description = "Activity tracker crn for COS bucket. Only required if 'create_cos_bucket' is true."
  default     = null
}

variable "sysdig_crn" {
  type        = string
  description = "Sysdig Monitoring crn for COS bucket. Only required if 'create_cos_bucket' is true."
  default     = null
}

buckets submodule is missing the ability to set the endpoint type to use

The root level module has a variable called management_endpoint_type_for_bucket which can be set to "public", "private", or "direct", however there doesn't seem to be any way to set this when creating a bucket using the buckets submodule. That means all buckets get created with "public" since this is the default value in the root level module

Add support for One-rate plan

Description

The variable cos_plan only allows standard and lite plans however it seems there is a third plan - "One Rate":

The One-rate plan offers large enterprises and ISVs (with multiple divisions or end-users) a predictable TCO based on one flat rate. The flat rate has built-in allowances for outbound bandwidth and operational requests. These allowances depend on the amount of storage capacity aggregated across multiple instances within a One Rate pricing region. There is no data retrieval charge.

New or affected modules

https://github.com/terraform-ibm-modules/terraform-ibm-cos

---

By submitting this issue, you agree to follow our Code of Conduct

Optionally create the IAM Auth policy (COS -> KMS)

Description

What:

• Give more control on whether the ibm_iam_authorization_policy (new input variable) object is created
• Current logic at https://github.com/terraform-ibm-modules/terraform-ibm-cos/blob/main/main.tf#L61
• Needed to cover scenario where IAM auth policies are created in advance at the account level - typically at account creation

Automated test suites for CBR rules

Description

Extracted out from original issue at #201 .

See whether we could expend the automated test suite as well to catch this type of issue - eg: trying to list the cos bucket in the instance as part of the test executing the complete example would catch this type of problem.

Allow naming of cos instance created by terraform-ibm-cos

Description

Most of the time we want to create a Cloud Object Storage instance with a predetermined name that adheres to a projects naming conventions.

The current behaviour when creating a COS instance is to use the name environment-name-cos.

New or affected modules

https://github.com/terraform-ibm-modules/terraform-ibm-cos

Other considerations

Existing consumers of this module that specify both create-cos-instance=true and cos-instance-name may see a change in behaviour. The current behaviour is to silently ignore the cos-instance-name provided. The new behaviour will be to change the name of the COS instance, which may result in an attempt to delete and re-create the instance.

Support for attaching access tag

Description

Support for attaching access tag

New or affected modules

---

By submitting this issue, you agree to follow our Code of Conduct

Review all COS DA related documentation

The COS DA has 3 flavors:

• instance
• secure-cross-regional-bucket
• secure-regional-bucket

The following items for all of these DA flavors will need to be reviewed and updated where required:

• DA variable descriptions (in each of the DA flavor's variables.tf files)
• The information in the ibm_catalog.json (labels, descriptions, features etc). Are we happy with the flavor naming?

Add support for cross region buckets to FSCloud profile submodule

The module at https://github.com/terraform-ibm-modules/terraform-ibm-cos/tree/main/profiles/fscloud only has support for regional buckets. Cross-regional buckets are also FSCloud compliant (and actually preferred when possible)

CBR rules are not applied to bucket and instance (incorrect ids passed in the rules)

Scenario to reproduce the issue:

1. Execute the complete examples at https://github.com/terraform-ibm-modules/terraform-ibm-cos/tree/main/examples/complete - this example creates 3 CBR rules (one for instance, and one per bucket) that are scoped to allow access only from a VPC (created in the example as well)
2. Change the CBR enforcement mode to 'enabled' (example set it to reporting by default)
3. Navigate to the COS instance from your local browser (so that you're not in the VPC..) - from the IBM cloud console for instance.

Result: Notice you can see the instance and buckets
Expected: Access denied message when accessing the instance as requests are not coming from within VPC (but the internet in this case). As below (displayed when fixing the CBR rule manually)

Issue is that the module pass the instance id as input, instead of the guid. https://github.com/terraform-ibm-modules/terraform-ibm-cos/blob/main/main.tf#L236

Affected modules

• terraform-ibm-cos
• Seeing the same problem across all modules using the same CBR code - so it would be good to do a sweep around

See whether we could expend the automated test suite as well to catch this type of issue - eg: trying to list the cos bucket in the instance as part of the test executing the complete example would catch this type of problem.

Create COS bucket replication submodule

Extract out the replication logic at https://github.com/terraform-ibm-modules/terraform-ibm-cos/tree/main/examples/replication into its own terraform submodule that consumers can consume

Add support to COS module to create IAM based resource key

The COS module should support creating IAM as well as HMAC resource keys. Currently it only supports HMAC keys however IAM API Keys are the preferred method of authentication for COS as per docs.

Proposal:

Rename these variables (both root level module and fscloud submodule)

• create_hmac_key -> create_resource_key
• hmac_key_name -> resource_key_name
• hmac_key_role -> resource_key_role

Create a new boolean variable called generate-hmac-credentials which should be used like this:

parameters = {
    "serviceid_crn" = var.resource_key_existing_serviceid_crn
    "HMAC"          = var.generate-hmac-credentials
  }

Since IAM is the recommended key type, I think we should set generate-hmac-credentials to false by default (make sure we mention this in the release notes, as it will attempt to destroy any existing hmac credentials and replace it with an IAM credential)

Expose lock capability on buckets

Description

1. Expose https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/cos_bucket#object_lock
2. Add a static check in the module to ensure object versioning is enabled when object lock is enabled
3. (May be done in a subsequent PR) - Expose object lock configuration https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/cos_bucket_object_lock_configuration

Allow user to configure the storage class

Description

Currently "standard" is hardcoded in the code.

storage_class

New or affected modules

• terraform-ibm-cos

---

By submitting this issue, you agree to follow our Code of Conduct

bucket crn and id outputs are list of 1 object

bucket_crn and bucket_id are impacted (at first glance, maybe more).

terraform-ibm-cos/outputs.tf

Line 19 in 753a10a

output "bucket_id" {

Suggest to make them string - otherwise caller need to specify first index - module.cos_instance_bucket.bucket_crn[0] in their code which is untidy.

Expose COS resource key details as outputs

The COS module supports creating a resource key, however it does not output it in its outputs.
We should add 3 new outputs to output the name, ID and value of the key.

The problem we will face is when we output the key value as a sensitive value, the buckets module complains because it has the following output:

output "buckets" {
  description = "Map of buckets created in the Cloud Object Storage Instance"
  value       = module.buckets
}

The output will now contain a sensitive value and hence terraform says it should be marked as a sensitive, which is not what we want. We should update the logic here so it only outputs bucket info. There is no need to include the resource key output in this output.

Ensure we also add the outputs to the fscloud submodule.

Expose cos instance guid as an output

Description

Expose cos instance guid as an output

New or affected modules

https://github.com/terraform-ibm-modules/terraform-ibm-cos

---

By submitting this issue, you agree to follow our Code of Conduct

Create a simple basic example for the module

Description

All example and the usage section go into a lot of complexity in term of replication, encryption, etc.

It would be desirable to have a simple, basic example that creates an instance with a bucket with the minimum amount of inputs to set.

Option to append a random suffix / previx to cos bucket name

Description

It would be good to have such an option to avoid common scenarios where we end up with clash in bucket name (with existing bucket name)

Expose the `add_bucket_name_suffix` option is buckets and fscloud submodules

Looks like its only possible to set the add_bucket_name_suffix in the root level module. Can we expose this in the buckets and fscloud submodules as an optional attribute to the buckets config variable (default to false).

Add ability to pass a list of buckets to create

Description

Allow to pass a list of buckets as input - loop within the module on creating that list.
Should work with creating a cos instance, or pointing to existing cos instance

New or affected modules

• terraform-ibm-cos

---

By submitting this issue, you agree to follow our Code of Conduct

Authorization COS - KMS malformed

Hello there,

We are facing an issue with your module, especially regarding authorization as shown below:

 ibm_resource_instance.kms_instance: Creating...
 2023/01/27 22:03:52 Terraform apply | module.cos.ibm_resource_instance.cos_instance[0]: Creating...
 2023/01/27 22:04:02 Terraform apply | ibm_resource_instance.kms_instance: Still creating... [10s elapsed]
 2023/01/27 22:04:02 Terraform apply | module.cos.ibm_resource_instance.cos_instance[0]: Still creating... [10s elapsed]
 2023/01/27 22:04:06 Terraform apply | ibm_resource_instance.kms_instance: Creation complete after 14s [id=crn:v1:bluemix:public:kms:eu-fr2:a/3f2cae45b0644f6d87aefcf404f5987f:ba24f111-f80e-4298-a301-ca20371a0d8d::]
 2023/01/27 22:04:06 Terraform apply | data.ibm_kms_keys.encryption_data: Reading...
 2023/01/27 22:04:06 Terraform apply | ibm_kms_key.encryption_key: Creating...
 2023/01/27 22:04:07 Terraform apply | data.ibm_kms_keys.encryption_data: Read complete after 0s [id=ba24f111-f80e-4298-a301-ca20371a0d8d]
 2023/01/27 22:04:07 Terraform apply | ibm_kms_key.encryption_key: Creation complete after 1s [id=crn:v1:bluemix:public:kms:eu-fr2:a/3f2cae45b0644f6d87aefcf404f5987f:ba24f111-f80e-4298-a301-ca20371a0d8d:key:8c87060a-6993-47b5-9ca6-1e2738973927]
 2023/01/27 22:04:08 Terraform apply | module.cos.ibm_resource_instance.cos_instance[0]: Creation complete after 16s [id=crn:v1:bluemix:public:cloud-object-storage:global:a/3f2cae45b0644f6d87aefcf404f5987f:d20bf547-6b85-4f4b-877c-1475c4bde1ee::]
 2023/01/27 22:04:08 Terraform apply | module.cos.ibm_resource_key.resource_key[0]: Creating...
 2023/01/27 22:04:08 Terraform apply | module.cos.ibm_iam_authorization_policy.policy[0]: Creating...
 2023/01/27 22:04:09 Terraform apply | module.cos.ibm_iam_authorization_policy.policy[0]: Creation complete after 1s [id=a77acd3f-d3c4-4c63-b1b5-0951bfee108d]
 2023/01/27 22:04:09 Terraform apply | module.cos.ibm_cos_bucket.cos_bucket[0]: Creating...
 2023/01/27 22:04:12 Terraform apply | module.cos.ibm_resource_key.resource_key[0]: Creation complete after 4s [id=crn:v1:bluemix:public:cloud-object-storage:global:a/3f2cae45b0644f6d87aefcf404f5987f:d20bf547-6b85-4f4b-877c-1475c4bde1ee:resource-key:3c34a882-8178-4ab8-b136-c010847e62b5]
 2023/01/27 22:04:12 Terraform apply | 
 2023/01/27 22:04:12 Terraform apply | Error: ServiceNotAuthorized: The specified COS Service Instance does not have sufficient permissions to access the resource associated with the KMS key CRN.
 2023/01/27 22:04:12 Terraform apply | 	status code: 401, request id: d32b07a0-e350-4e6a-aa86-787e467c0b4f, host id: 
 2023/01/27 22:04:12 Terraform apply | 
 2023/01/27 22:04:12 Terraform apply |   with module.cos.ibm_cos_bucket.cos_bucket[0],
 2023/01/27 22:04:12 Terraform apply |   on .terraform/modules/cos/main.tf line 76, in resource "ibm_cos_bucket" "cos_bucket":
 2023/01/27 22:04:12 Terraform apply |   76: resource "ibm_cos_bucket" "cos_bucket" {
 2023/01/27 22:04:12 Terraform apply | 
 2023/01/27 22:04:12 �[1m�[31mTerraform APPLY error: Terraform APPLY errorexit status 1�[39m�[0m

We found out that your authorization resource input for the source_service_id, is not correct, it should be the GUID of the COS instance instead of ID.

It results into this authorization in your IAM:

###Proposal

locals {
 cos_instance_id      = var.create_cos_instance == true ? tolist(ibm_resource_instance.cos_instance[*].id)[0] : var.existing_cos_instance_id
 cos_instance_guid      = var.create_cos_instance == true ? : tolist(ibm_resource_instance.cos_instance[*].guid)[0] : element(split(":",var.existing_cos_instance_id),length(split(":",var.existing_cos_instance_id)-3)
 create_access_policy = var.encryption_enabled && var.create_cos_instance
}

# Create IAM Access Policy to allow Key protect to access COS instance
resource "ibm_iam_authorization_policy" "policy" {
 count                       = local.create_access_policy ? 1 : 0
 source_service_name         = "cloud-object-storage"
 source_resource_instance_id = local.cos_instance_guid
 target_service_name         = "kms"
 target_resource_instance_id = var.existing_key_protect_instance_guid
 roles                       = ["Reader"]
}

Resulting into a correct Authorization:

Let me know if you want me to do a PR.

Regards

Complete example: Network zone contains all VPC to the CBR rule

Issue reference to the CBR rule in the 'complete' example at https://github.com/terraform-ibm-modules/terraform-ibm-cos/tree/main/examples/complete

I executed the example multiple times. When looking at the rule in the UI - I see many more VPCs that the one VPC created in the example.

Affected modules

• terraform-ibm-cos

Timing issue with auth policy when creating multiple buckets with buckets submodule

When you are using the buckets or fscloud submodule, and you are creating more than 1 bucket with KMS encryption enabled, and have skip_iam_authorization_policy set to false for 1 bucket and set to true for the other, there is a timing issue where 1 bucket config will start to create the KMS auth policy first, while the other will start to create the bucket first, but bucket creation will fail because the auth policy may not yet be created.

We found this issue in terraform-ibm-modules/terraform-ibm-observability-da#8

Ideally we need to find a way where if any bucket config is set to create an auth policy, NO buckets should get created until its created. This logic would have to live in the buckets submodule I guess.

We probably should also fast fail if we detect that multiple buckets have skip_iam_authorization_policy set to false since its going to try and create duplicate policies and fail anyway.

Cannot create COS instance only with skip_iam_authorization_policy=false in v6.0.0

Affected modules

• COS

Terraform CLI and Terraform provider versions

• Terraform version: v1.4.5
• Provider version: v1.51.0

Expected behavior

As a user I should be able to create a COS instance only without buckets and associate it to a KMS instance.

The ibm_iam_authorization_policy to authorize COS instance to read KMS keys should not be skipped.

I need something similar to this:
module "cos_instance" {
source = "../../"
cos_instance_name = "${var.prefix}-cos"
create_cos_bucket = false
resource_group_id = module.resource_group.resource_group_id
existing_kms_instance_guid = module.key_protect_all_inclusive.key_protect_guid
region = var.region
cross_region_location = null
activity_tracker_crn = null
resource_key_existing_serviceid_crn = ibm_iam_service_id.resource_key_existing_serviceid.crn
skip_iam_authorization_policy = false
}

Actual behavior

I get the following error:
Error: Invalid combination of arguments

with module.cos_module["vpc-cos"].ibm_iam_authorization_policy.policy[0],
on .terraform/modules/cos_module/main.tf line 73, in resource "ibm_iam_authorization_policy" "policy":
73: target_service_name = local.kms_service

"target_service_name": one of resource_attributes,target_service_name must be specified

Steps to reproduce (including links and screen captures)

By submitting this issue, you agree to follow our Code of Conduct

fscloud profile for COS (instance + buckets)

Description

Tracking ongoing work at https://github.com/terraform-ibm-modules/terraform-ibm-cos/tree/fscloud

---

By submitting this issue, you agree to follow our Code of Conduct

Double check that access tags are indeed evaluated in the CBR rules

Description

There is a need to have an actual check that the tags passed in the CBR rule are indeed used. So far, I think the assumption is that they are, but I don't think this has been actually verified. Perhaps, as part of this add a tag example here: https://github.com/terraform-ibm-modules/terraform-ibm-cos/tree/main/examples

New or affected modules

• terraform-ibm-cos

---

By submitting this issue, you agree to follow our Code of Conduct

Remove `kms_encryption_enabled` attribute from var.bucket_configs in FSCloud COS module

FSCloud mandates KMS encryption, so we should expose the option kms_encryption_enabled in the bucket_configs in FSCloud submodule. Instead default it to true in here

NOTE: Some of the bucket_validations will need to be updated as part of this change

Review COS DA diagrams

Review the diagram(s) in the reference-architectures directory...

• Do they contain accurate relevant information
• Are they using approved icons / objects?

Release Pipeline failed

For details see: https://github.com/terraform-ibm-modules/terraform-ibm-cos/actions/runs/8580148041

FSCloud submodule usage is incorrect

The usage is wrong https://github.com/terraform-ibm-modules/terraform-ibm-cos/tree/main/modules/fscloud#usage

Enable the passing of a list of resource key to create

Description

Current logic creates one single key (optionally) at

terraform-ibm-cos/main.tf

Line 73 in 39854bc

resource "ibm_resource_key" "resource_key" {

.

It would be desirable to allow creating multiple keys with different roles

terraform-ibm-cos/main.tf

Line 81 in 39854bc

role = var.resource_key_role

. Essentially allow the passing of a list of roles as input to the module instead of a single role.

Allow the COS bucket to configure allowed_ip

Description

Allow the capability to configure allowed_ip for the cos bucket in the main.tf

argument here: https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/cos_bucket#allowed_ip

New or affected modules

COS module: https://github.com/terraform-ibm-modules/terraform-ibm-cos/blob/main/main.tf#L127

---

By submitting this issue, you agree to follow our Code of Conduct

Document CBR UI tag limitation in module

Description

The CBR UI does not currently display properly tags. The question came from one of the module consumer, and is likely to repeat, so would be good to document directly into the module README.md.

New or affected modules

• terraform-ibm-cos

---

By submitting this issue, you agree to follow our Code of Conduct

Don't output the bucket name until bucket is created

Can we have the module only output the bucket name once the bucket is actually created and ready for use? That way consuming modules won't have to add explicit depends_on in their code since they always take bucket name is as input as opposed to bucket ID.

To do this, we can add a depends_on in the outout:

output "bucket_name" {
  description = "Bucket name"
  value       = local.bucket_name
  depends_on  = [local.bucket_id]
}

Optional Bucket Creation

Description

Most of the time we want to create a bucket when creating a COS instance. But this is not always the case. Make bucket creation optional

New or affected modules

https://github.com/terraform-ibm-modules/terraform-ibm-cos

---

By submitting this issue, you agree to follow our Code of Conduct

Missing bucket_endpoint in version 6.0.0

bucket_endpoint was removed in version 6.0.0. However, this value is used by the provider to construct the url used to access cos management apis.

https://github.com/IBM-Cloud/terraform-provider-ibm/blob/master/ibm/service/cos/resource_ibm_cos_bucket.go#L1140

retention policy is set to enabled by default in root module

terraform-ibm-cos/variables.tf

Line 135 in 753a10a

default = true

Suggest to set to false as this is going to prevent from deleting objects from cos bucket. Leave the retention enable only in the fscloud profile version at https://github.com/terraform-ibm-modules/terraform-ibm-cos/tree/main/profiles/fscloud

endpoint_type parameter is missing from cos_bucket

endpoint_type should be defined in cos_bucket as it is defined in cos_bucket1. When the user specifies an endpoint_type along with enabling encryption on their cos_bucket the endpoint_type will not take affect.

Affected modules

• https://github.com/terraform-ibm-modules/terraform-ibm-cos

Expose direct endpoint in the output

Description

This module currently exposes the public and private endpoint in the output. But there are also the direct endpoints that are available from COS. Could this be added to the output as well, please?

New or affected modules

• terraform-ibm-cos

---

By submitting this issue, you agree to follow our Code of Conduct