Custom extension for sending metric events from Apache Druid to Google Stackdriver.
Run mvn package
to create a .jar
file that contains all dependencies.
The generated .jar
file needs to be copied into druids extensions/
directory (for example into /app/druid/extensions/stackdriver-emitter/
).
When running druid it should automatically detect and execute this extension.
To authenticate to the Stackdriver API the GOOGLE_APPLICATION_CREDENTIALS
environment variable needs to be set pointing to the .json
file with the necessary credentials.
To set these credentials in GKE the following steps are necessary:
- Create service account keys which should download a
.json
with the credentials - Create a Kubernetes secret resource for those service account credentials:
kubectl create secret generic service-account-credentials --from-file /path/to/credentials.json
- Mount the credentials in the container that needs access
- Set the
GOOGLE_APPLICATION_CREDENTIALS
environment variable
Configuration parameters for the plugin that can be set in the config file (eg. common.runtime.properties
):
druid.emitter=stackdriver
# number of events sent together in one request to Stackdriver
druid.emitter.stackdriver.flushThreshold=5
# max. queue size for event queue
druid.emitter.stackdriver.maxQueueSize=1000
# frequency in ms of sending collected events to Stackdriver
druid.emitter.stackdriver.consumeDelay=60000
# Stackdriver project ID
druid.emitter.stackdriver.projectId=streaming-events-dev
# optionally, file with whitelist of druid events that should be processed
druid.emitter.stackdriver.metricMapPath=/app/druid/extensions/stackdriver-emitter/defaultMetrics.json
The extension must also be registered in the config file:
druid.extensions.loadList=[..., "stackdriver-emitter", ...]
The emitted metric events should appear in Stackdriver: