terraform-ibm-modules/terraform-ibm-icd-elasticsearch

Implements an instance of the IBM Cloud Databases for Elasticsearch service.

ICD Elasticsearch Module

Overview

• terraform-ibm-icd-elasticsearch
• Submodules
  • fscloud
• Examples
  • Basic example
  • Complete example with autoscaling, BYOK encryption, service credentials creation, index creation and updates to cluster-wide settings
  • Financial Services Cloud profile example with autoscaling enabled
• Contributing

This module implements an instance of the IBM Cloud Databases for Elasticsearch service.

Usage

provider "ibm" {
  ibmcloud_api_key = "XXXXXXXXXXXXXX"
  region           = "us-south"
}

module "icd_elasticsearch" {
  source            = "terraform-ibm-modules/icd-elasticsearch/ibm"
  version           = "X.X.X"  # Replace "X.X.X" with a release version to lock into a specific release
  resource_group_id = "xxXXxxXXxXxXXXXxxXxxxXXXXxXXXXX"
  region            = "us-south"
}

Required IAM access policies

You need the following permissions to run this module.

• Account Management
  • Databases for Elasticsearch service
    • Editor role access

Requirements

Name	Version
terraform	>= 1.3.0, < 1.7.0
ibm	>= 1.62.0, <2.0.0
time	>= 0.9.1

Modules

Name	Source	Version
cbr_rule	terraform-ibm-modules/cbr/ibm//modules/cbr-rule-module	1.22.0

Resources

Name	Type
ibm_database.elasticsearch	resource
ibm_iam_authorization_policy.policy	resource
ibm_resource_key.service_credentials	resource
ibm_resource_tag.elasticsearch_tag	resource
time_sleep.wait_for_authorization_policy	resource
ibm_database_connection.database_connection	data source

Inputs

Name	Description	Type	Default	Required
access_tags	A list of access tags to apply to the Elasticsearch instance created by the module, see https://cloud.ibm.com/docs/account?topic=account-access-tags-tutorial for more details	list(string)	[]	no
admin_pass	The password for the database administrator. If the admin password is null then the admin user ID cannot be accessed. More users can be specified in a user block.	string	null	no
auto_scaling	Optional rules to allow the database to increase resources in response to usage. Only a single autoscaling block is allowed. Make sure you understand the effects of autoscaling, especially for production environments. See https://cloud.ibm.com/docs/databases-for-elasticsearch?topic=databases-for-elasticsearch-autoscaling in the IBM Cloud Docs.	object({ disk = object({ capacity_enabled = optional(bool, false) free_space_less_than_percent = optional(number, 10) io_above_percent = optional(number, 90) io_enabled = optional(bool, false) io_over_period = optional(string, "15m") rate_increase_percent = optional(number, 10) rate_limit_mb_per_member = optional(number, 3670016) rate_period_seconds = optional(number, 900) rate_units = optional(string, "mb") }) memory = object({ io_above_percent = optional(number, 90) io_enabled = optional(bool, false) io_over_period = optional(string, "15m") rate_increase_percent = optional(number, 10) rate_limit_mb_per_member = optional(number, 114688) rate_period_seconds = optional(number, 900) rate_units = optional(string, "mb") }) })	null	no
backup_crn	The CRN of a backup resource to restore from. The backup is created by a database deployment with the same service ID. The backup is loaded after provisioning and the new deployment starts up that uses that data. A backup CRN is in the format crn:v1:<…>:backup:. If omitted, the database is provisioned empty.	string	null	no
backup_encryption_key_crn	The CRN of a Key Protect key, that you want to use for encrypting disk that holds deployment backups. Only used if var.kms_encryption_enabled is set to true. If no value is passed here, the value passed for the 'kms_key_crn' variable is used, unless 'use_default_backup_encryption_key' is set to 'true'. BYOK for backups is available only in US regions us-south and us-east, and eu-de. Only keys in the us-south and eu-de are durable to region failures. To ensure that your backups are available even if a region failure occurs, you must use a key from us-south or eu-de. Take note that Hyper Protect Crypto Services for IBM Cloud® Databases backups is not currently supported, so if no value is passed here, but a HPCS value is passed for var.kms_key_crn, databases backup encryption will use the default encryption keys.	string	null	no
cbr_rules	(Optional, list) List of CBR rules to create	list(object({ description = string account_id = string rule_contexts = list(object({ attributes = optional(list(object({ name = string value = string }))) })) enforcement_mode = string }))	[]	no
elasticsearch_version	Version of Elasticsearch to deploy, must be 8.7 or 8.10 (Enterprise plan only or Platinum if 8.10 or above). If no value passed, the current ICD preferred version is used.	string	null	no
existing_kms_instance_guid	The GUID of the Hyper Protect or Key Protect instance in which the key specified in var.kms_key_crn and var.backup_encryption_key_crn is coming from. Only required if var.kms_encryption_enabled is 'true', var.skip_iam_authorization_policy is 'false', and passing a value for var.kms_key_crn and/or var.backup_encryption_key_crn.	string	null	no
kms_encryption_enabled	Set this to true to control the encryption keys used to encrypt the data that you store in IBM Cloud® Databases. If set to false, the data is encrypted by using randomly generated keys. For more info on Key Protect integration, see https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-key-protect. For more info on HPCS integration, see https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-hpcs	bool	false	no
kms_key_crn	The root key CRN of a Key Management Services like Key Protect or Hyper Protect Crypto Service (HPCS) that you want to use for disk encryption. Only used if var.kms_encryption_enabled is set to true.	string	null	no
member_cpu_count	Allocated dedicated CPU per member. For shared CPU, set to 0. For more information, see https://cloud.ibm.com/docs/databases-for-elasticsearch?topic=databases-for-elasticsearch-resources-scaling	number	0	no
member_disk_mb	Allocated disk per-member. For more information, see https://cloud.ibm.com/docs/databases-for-elasticsearch?topic=databases-for-elasticsearch-resources-scaling	number	5120	no
member_memory_mb	Allocated memory per-member. For more information, see https://cloud.ibm.com/docs/databases-for-elasticsearch?topic=databases-for-elasticsearch-resources-scaling	number	1024	no
members	Allocated number of members. For more information, see https://cloud.ibm.com/docs/databases-for-elasticsearch?topic=databases-for-elasticsearch-resources-scaling	number	3	no
name	The name to give the Elasticsearch instance.	string	n/a	yes
plan	The name of the service plan that you choose for your Elasticsearch instance	string	"enterprise"	no
region	The region where you want to deploy your instance.	string	"us-south"	no
resource_group_id	The resource group ID where the Elasticsearch instance will be created.	string	n/a	yes
service_credential_names	Map of name, role for service credentials that you want to create for the database	map(string)	{}	no
service_endpoints	Sets the endpoint of the instance, valid values are 'public', 'private', or 'public-and-private'	string	"public"	no
skip_iam_authorization_policy	Set to true to skip the creation of an IAM authorization policy that permits all Elasticsearch database instances in the resource group to read the encryption key from the KMS instance. If set to false, pass in a value for the KMS instance in the existing_kms_instance_guid variable. In addition, no policy is created if var.kms_encryption_enabled is set to false.	bool	false	no
tags	Tags associated with the instance (Optional, array of strings).	list(string)	[]	no
use_default_backup_encryption_key	Set to true to use default ICD randomly generated keys.	bool	false	no
users	A list of users that you want to create on the database. Multiple blocks are allowed. The user password must be in the range of 10-32 characters. Be warned that in most case using IAM service credentials (via the var.service_credential_names) is sufficient to control access to the Elasticsearch instance. This blocks creates native elasticsearch database users, more info on that can be found here https://cloud.ibm.com/docs/databases-for-elasticsearch?topic=databases-for-elasticsearch-connection-strings&interface=cli#connection-strings-add-users	list(object({ name = string password = string # pragma: allowlist secret type = string # "type" is required to generate the connection string for the outputs. role = optional(string) }))	[]	no

Outputs

Name	Description
adminuser	Database admin user name
cbr_rule_ids	CBR rule ids created to restrict Elasticsearch
certificate_base64	Database connection certificate
crn	Elasticsearch instance crn
guid	Elasticsearch instance guid
hostname	Database connection hostname
id	Elasticsearch id
port	Database connection port
service_credentials_json	Service credentials json map
service_credentials_object	Service credentials object
version	Elasticsearch version

Contributing

You can report issues and request features for this module in GitHub issues in the module repo. See Report an issue or request a feature.

To set up your local development environment, see Local development setup in the project documentation.

terraform-ibm-icd-elasticsearch's Issues

Review Elasticsearch DA diagram

Review the diagram(s) in the reference-architectures directory...

• Do they contain accurate relevant information
• Are they using approved icons / objects?

Add support to module for index creation and cluster configuration

The complete example shows how to use the elasticsearch provider to create indexes and configure cluster settings. Can we add that support directly to the module?

Review all Elasticsearch DA related documentation

The Elasticsearch DA has 1 flavor currently:

• secure

The following items will need to be reviewed and updated where required:

• DA variable descriptions (in the DA's variables.tf file)
• The information in the ibm_catalog.json (labels, descriptions, features etc). Are we happy with the flavor naming?

No valid category tag added for Elasticsearch DA

There needs to be a valid category tag added in ibm_catalog.json. For list of valid tags, you can run ibmcloud catalog filter options --all

Looks like databases needs to be updated to database:

No permissions are showing for Elasticsearch DA

No permissions are showing for Elasticsearch DA. Please ensure that the CRNs are correct in ibm_catalog.json

Include configuration for vector database

< Placeholder for now > details to come