Issues consuming to consume messages after creating some ACLs

[confluentinc]

Issues consuming to consume messages after creating some ACLs

[linouk23]

That's interesting, I think it depends on a use case and it might be OK to share the same producer / consumers for multiple topics. If you do want to create one-to-one mapping then for_each meta-argument might help. Feel free to open another issue if you need a little bit more details about this question.

[linouk23]

@VipulZopSmart here's a full set of ACLs you need:

resource "confluentcloud_kafka_acl" "app-producer-write-to-topic" {
  kafka_cluster = confluentcloud_kafka_cluster.basic.id
  resource_type = "TOPIC"
  resource_name = confluentcloud_kafka_topic.orders.topic_name
  pattern_type  = "LITERAL"
  principal     = "User:${confluentcloud_service_account.app-producer.id}"
  operation     = "WRITE"
  permission    = "ALLOW"
  http_endpoint = confluentcloud_kafka_cluster.basic.http_endpoint
  credentials {
    key    = confluentcloud_api_key.app-manager-kafka-api-key.id
    secret = confluentcloud_api_key.app-manager-kafka-api-key.secret
  }
}

// Note that in order to consume from a topic, the principal of the consumer ('app-consumer' service account)
// needs to be authorized to perform 'READ' operation on both Topic and Group resources:
// confluentcloud_kafka_acl.app-consumer-read-on-topic, confluentcloud_kafka_acl.app-consumer-read-on-group.
// https://docs.confluent.io/platform/current/kafka/authorization.html#using-acls
resource "confluentcloud_kafka_acl" "app-consumer-read-on-topic" {
  kafka_cluster = confluentcloud_kafka_cluster.basic.id
  resource_type = "TOPIC"
  resource_name = confluentcloud_kafka_topic.orders.topic_name
  pattern_type  = "LITERAL"
  principal     = "User:${confluentcloud_service_account.app-consumer.id}"
  operation     = "READ"
  permission    = "ALLOW"
  http_endpoint = confluentcloud_kafka_cluster.basic.http_endpoint
  credentials {
    key    = confluentcloud_api_key.app-manager-kafka-api-key.id
    secret = confluentcloud_api_key.app-manager-kafka-api-key.secret
  }
}

resource "confluentcloud_kafka_acl" "app-consumer-read-on-group" {
  kafka_cluster = confluentcloud_kafka_cluster.basic.id
  resource_type = "GROUP"
  // The existing values of resource_name, pattern_type attributes are set up to match Confluent CLI's default consumer group ID ("confluent_cli_consumer_<uuid>").
  // https://docs.confluent.io/confluent-cli/current/command-reference/kafka/topic/confluent_kafka_topic_consume.html
  // Update the values of resource_name, pattern_type attributes to match your target consumer group ID.
  // https://docs.confluent.io/platform/current/kafka/authorization.html#prefixed-acls
  resource_name = "confluent_cli_consumer_"
  pattern_type  = "PREFIXED"
  principal     = "User:${confluentcloud_service_account.app-consumer.id}"
  operation     = "READ"
  permission    = "ALLOW"
  http_endpoint = confluentcloud_kafka_cluster.basic.http_endpoint
  credentials {
    key    = confluentcloud_api_key.app-manager-kafka-api-key.id
    secret = confluentcloud_api_key.app-manager-kafka-api-key.secret
  }
}

Let me know if it helps!

[VipulZopSmart]

Thanks, it worked for that. One thing, how should i generate service accounts(for producers and consumers) from a array of topics and i think there should be only one producer and one consumer for each topic.