AWS Glue Schema Registry compatability

[tomellis91]

AWS Glue Schema Registry compatability

[sfc-gh-gjachimko]

Let me check it with the devs and we will get back to you.

[sfc-gh-gjachimko]

Hi
So - it is possible to configure our plugin with AWS Glue.
There some additional steps required though - you need to bundle the plugin with the proper JAR containing valid converter. You didn't specify which data format you'd like to use, so here are the options:

• https://mvnrepository.com/artifact/software.amazon.glue/jsonschema-kafkaconnect-converter
• https://mvnrepository.com/artifact/software.amazon.glue/protobuf-kafkaconnect-converter
• https://mvnrepository.com/artifact/software.amazon.glue/schema-registry-kafkaconnect-converter

Having these two pieces bundled together, please follow AWS guideline on creating plugin: https://docs.aws.amazon.com/msk/latest/developerguide/msk-connect-plugins.html

Setting up the MSK connector will require proper IAM role - https://docs.aws.amazon.com/msk/latest/developerguide/mkc-tutorial-setup.html.

Then, you will need to configure MSK worker and connector.

From the connector perspective, the important piece of functionality to make it work with AWS is the following:

value.converter.region=[aws region]
value.converter.schemaAutoRegistrationEnabled=true
value.converter.dataFormat=[your format specifier]
value.converter.registry.name=[AWS Glue Registry Name]

In case of further issues, please feel free to reach out or contact our support.

[tomellis91]

thank you @sfc-gh-gjachimko 🙌 I will try this out.