pannoi/kafka-schema-operator

Operator to provide Kafka Schema in declarative way using KubernetesCustomResources

Kafka Schema Operator

The Kafka Schema Operator delivers as easy way to deliver Kafka Schemas declaratively via Kubernetes CRD

Schema First approach should be implemented if you are using schema-registry according to best practices

Operator schema registry compatibility

• Confluent
• Strimzi

Operator features

• Create schemas declaratively via CRD
• Automatic schema versions update
• GitOps (automatic updates)

Installation

helm repo add kafka-schema-operator https://pannoi.github.io/kafka-schema-operator-helm/
helm repo update
helm upgrade --install kafka-schema-operator kafka-schema-operator/kafka-schema-operator --values values.yaml

You need to set SCHEMA_REGSITRY_HOST and SCHEMA_REGSITRY_PORT initial functionality

schemaRegistry:
  host:
  port:

If your schema registry needs API authentication add Key and Secret values in addition to host and port

schemaRegistry:
  apiKey:
  apiSecret:

You can refer from secretRef

For more values check default values

How to use

Deploy ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  name: kafka-schema
  namespace: default
data:
  schema: |
    {
      "namespace": "testing",
      "type": "record",
      "name": "testing",
      "fields": [
        {"name": "id", "type": "string"},
        {"name": "email", "type": "string"}
      ]
    }

Refer to created ConfigMap via CustomResource KafkaSchema

apiVersion: kafka-schema-operator.pannoi/v1beta1
kind: KafkaSchema
metadata:
  name: kafka-schema
  namespace: default
spec:
  name: testing
  schemaSerializer: string
  autoReconciliation: true # true = autoUpdate schema, false = for update CR should be re-created (not set => false)
  terminationProtection: true # true = don't delete resources on CR deletion, false = when CR deleted, deletes all resource: ConfigMap, Schema from registry (not set => false)
  data:
    configRef: kafka-schema # ConfigMap
    format: avro # avro/protobuf/json
    compatibility: # BACKWARD | BACKWARD_TRANSITIVE | FORWARD | FORWARD_TRANSITIVE | FULL | FULL_TRANSITIVE | NONE

Resources (KafkaSchema & ConfigMap) should be located in same namespace