apakulov/kafka-graphite

Graphite integration for Kafka

kafka-graphite's Issues

Configuring the prefix with a "." character makes the plugin not report anything to Graphite

I'm trying to configure the prefix with something like foo.bar instead of the default kafka, so I can categorize the metrics of the broker. The plugin picks it up properly, as it shows in the log, but no metrics are sent to graphite at all.

[2018-01-29 13:27:56,003] INFO Configuring Kafka Graphite Reporter with host=172.28.128.10, port=2003, prefix=axim.kafka1 and include=null, exclude=null, jvm=true (kafka.metrics.KafkaGraphiteMetricsReporter)

Looking at the code in this repo, I don't see anything that could cause this... perhaps something inside the GraphiteReporter?

Confluent MetricsReporter support

I am able to collect the client metrics for kafka-connect service but for the confluent schema-registry when I enable the client metrics I get the following error

[2017-08-04 00:16:31,871] ERROR Server died unexpectedly: (io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain:51)
java.lang.RuntimeException: org.apache.kafka.common.metrics.GraphiteReporter is not an instance of io.confluent.common.metrics.MetricsReporter
at io.confluent.common.config.AbstractConfig.getConfiguredInstances(AbstractConfig.java:220)
at io.confluent.rest.Application.(Application.java:97)
at io.confluent.kafka.schemaregistry.rest.SchemaRegistryRestApplication.(SchemaRegistryRestApplication.java:47)
at io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain.main(SchemaRegistryMain.java:42)

How to enable client side metrics

I was trying to use GraphiteReporter to get kafka.consumer metrics in Graphite. No chance so far.

My setup is that I implemented a JAVA projcect of a simple consumer. Put this properties in class path:

metric.reporters=org.apache.kafka.common.metrics.GraphiteReporter
kafka.graphite.metrics.prefix=test
kafka.metrics.polling.interval.secs=1
kafka.graphite.metrics.reporter.enabled=true
kafka.graphite.metrics.host=10.0.0.10
kafka.graphite.metrics.port=200
kafka.graphite.metrics.jvm.enabled=false

I'm not sure how to get these metrics. From my search, kafka.consumer metrics have to be provided by client.

ConcurrentModificationException

Hi,

we're seeing a bunch of ConcurrentModificationException in our system and losing quite a bit of metrics:

2018-06-03T03:31:37.414-04:00 -- level=Warning, host=icd10gen-1, component=ICD10Generator, module=o.a.k.c.m.GraphiteReporter, version=1.0, message="Error writing to Graphite" ,exception= java.util.ConcurrentModificationException
        at java.util.LinkedHashMap$LinkedHashIterator.nextNode(LinkedHashMap.java:719)
        at java.util.LinkedHashMap$LinkedKeyIterator.next(LinkedHashMap.java:742)
        at java.util.AbstractCollection.addAll(AbstractCollection.java:343)
        at java.util.HashSet.<init>(HashSet.java:120)
        at org.apache.kafka.common.internals.PartitionStates.partitionSet(PartitionStates.java:65)
        at org.apache.kafka.clients.consumer.internals.SubscriptionState.assignedPartitions(SubscriptionState.java:298)
        at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$ConsumerCoordinatorMetrics$1.measure(ConsumerCoordinator.java:896)
        at org.apache.kafka.common.metrics.KafkaMetric.measurableValue(KafkaMetric.java:84)
        at org.apache.kafka.common.metrics.KafkaMetric.value(KafkaMetric.java:58)
        at org.apache.kafka.common.metrics.GraphiteReporter.run(GraphiteReporter.java:132)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

NoSuchMethod Exception using 0.10.0.0

I am trying to test using this metric reporter with a Kafka producer. Running 0.10.0.0.

Every time the producer starts up, I get this error regardless on if I set "kafka.graphite.metrics.reporter.enabled" to "true", "false", true, or false.

Has anyone else come across this or know a workaround?

Caused by: java.lang.NoSuchMethodError: org.apache.kafka.common.metrics.GraphiteReporter$GraphiteConfig.getBoolean(Ljava/lang/String;)Z at org.apache.kafka.common.metrics.GraphiteReporter.init(GraphiteReporter.java:76) at org.apache.kafka.common.metrics.Metrics.<init>(Metrics.java:129) at org.apache.kafka.common.metrics.Metrics.<init>(Metrics.java:111) at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:224)