Using configProviders other than file fails validation

[swehner]

Using configProviders other than file fails validation

[sfc-gh-xhuang]

Merged in #753

This will go out in the 2.2.0 release slated for January

[hitesh-yadav-x]

Assuming that you are trying to pull the credentials from AWS Secrets Manager. May I suggest use the default environment provider that kafka has and inject the variables in the container/run environment.

If using DOCKER set
ENV CONNECT_CONFIG_PROVIDERS="env"
ENV CONNECT_CONFIG_PROVIDERS_ENV_CLASS="org.apache.kafka.common.config.provider.EnvVarConfigProvider"

Then in config use below:

"snowflake.role.name": "${env:SNOWFLAKE_ROLE}",

[swehner]

Assuming that you are trying to pull the credentials from AWS Secrets Manager. May I suggest use the default environment provider that kafka has and inject the variables in the container/run environment.

If using DOCKER set ENV CONNECT_CONFIG_PROVIDERS="env" ENV CONNECT_CONFIG_PROVIDERS_ENV_CLASS="org.apache.kafka.common.config.provider.EnvVarConfigProvider"

Then in config use below:

"snowflake.role.name": "${env:SNOWFLAKE_ROLE}",

Thanks for the suggestions - I'm trying to pull them from property store (ssm), but secrets manager would be fine.
Problem is that i'm using msk connect as of now and I don't think I can set environment vars for the workers. I will keep it in mind once we switch over to a different cluster setup.

In any case isn't it a bit strange to restrict this specific validation behavior only to a configprovider named file?