entur/terraform-aiven-kafka-connect-elasticsearch-sink

Terraform module for managing Elasticsearch sink connector on Aiven managed KafkaConnect cluster

Elasticsearch sink connector

Terraform module for managing Elasticsearch sink connector on Aiven managed KafkaConnect cluster. This module depends on Aiven kafka init module to access basic information about Aiven's KafkaConnect cluster

Module

Example

Getting started

Access to Aiven terraform provider requires an API authentication token which can be generated from Aiven console Aiven authentication token can be provided as an environment variable with TF_VAR_ prefix or in a .tfvars file, otherwise from Harness Secrets Manager if you are provisioning from Harness.

Elasticsearch connector configuration

To be able to sink data to Elasticsearch server you need to figure out

• the topics on the Kafka cluster to sink data from
• the target Elasticsearch service URL(s)
• credentials(username and password) to get access to the Elasticsearch service
• Kafka user with permission to read topic schema

Example using the latest release

module "elasticsearch-sink" {
  source = "github.com/entur/terraform-aiven-kafka-connect-elasticsearch-sink//modules/elasticsearch-sink?ref=v0.1.3"
  ...
}

See the README.md under module's subfolder for a list of supported inputs and outputs. For examples showing how they're implemented, check the examples subfolder.

Version constraints

You can control the version of a module dependency by adding ?ref=TAG at the end of the source argument, as shown in the example above. This is highly recommended. You can find a list of available versions here.

Dependency automation tools such as Renovate Bot will be able to discover new releases and suggest updates automatically.

Reference(s)

• How to create an Elasticsearch sink connector on Aiven platform
• Git repo for Aiven's Elasticsearch sink connector

terraform-aiven-kafka-connect-elasticsearch-sink's Issues

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update terraform aiven to v4

Detected dependencies

github-actions

.github/workflows/pr-tests-terraform.yml

.github/workflows/pr-validation.yml

.github/workflows/release-please.yml

terraform

modules/elasticsearch-sink/versions.tf

• aiven >= 3.0.0, < 4.0.0
• google >=4.40.0
• hashicorp/terraform >=0.13.2

---

• Check this box to trigger a request for Renovate to run again on this repository