knoguchi / cm-druid Goto Github PK
View Code? Open in Web Editor NEWDruid service descriptor and parcel for Cloudera CDH5
Druid service descriptor and parcel for Cloudera CDH5
Hi,
The parcel page of my cloudera manager displays this for Druid.
Error for parcel DRUID-0.10.0-0.0.3-el7 : Parcel not available for OS Distribution RHEL7.
Is there something I can do?
Some configuration keys have dependencies or mutually exclusive.
As of now the supported OS is RedHat Enterprise 6 "el6" only that is defined in config.mk.
Add more OS and versions that CDH supports.
Possible distro suffixes
I did not have a chance to use your parcel yet, but looking at the code it appears that only MySQL is supported.
Could you also add support for PostgreSQL? Both, Druid and Cloudera can work with PostgreSQL.
Provide upgrade path.
The Druid roles must be rolling restarted at the configuration change or Druid upgrade.
Example:
Define versioning scheme for the following cases.
Define Release candidate tags and prod tag.
Parcels
There doesn't seem to be standard but most of Cloudera parcels follow this pattern:
<parcel name>-<original version>.cdh<cdh version>.<patch version>.<build version>
Example: the parcel version for druid-0.9.2, CDH 5.8.0 should be
DRUID-0.9.2.cdh.cdh5.8.0.p0.1
CSD
As far as I collected examples, they follow this pattern:
<service type>-<cdh version>
Example:
DRUID-5.8.0
But I'm not sure if this is expressive enough. I guess one service type can support one version only in CDH, hence the cdh version only. Needs research.
Unfortunately CDH5 supports IPv4 only. The CDH installation manual explicitly mentioned this.
However sometimes we don't have control over the IP stack configuration.
Many CDH services are started with -Djava.net.preferIPv4Stack=true
regardless the services that actually support IPv6. Hence, Druid failed to connect to Zookeeper if IPv6 is enabled.
Add the same config to Druid.
monitoring is essential for the operation and for the optimal resource allocation.
I would like to use Druid Emitters to push the metrics to CDH.
Metrics
http://druid.io/docs/latest/operations/metrics.html
Alerts
For now, please include necessary extensions under extensions
directory when you make the parcel. As of now the extensions are loaded from the /opt/parcels/DRUID directory. To make changes to the extension load list does not automatically pull the extensions and their dependencies.
I don't think it's feasible to create a giant parcel that includes all the extensions. More over I have a custom extension that is not part of Druid distribution. I need a way to "add on" extensions. pull-deps
can pull extensions from Maven repositories.
http://druid.io/docs/latest/operations/pull-deps.html
Where to save the extensions?
When you allocate roles to the servers. the wizard should recommend values.
When ever configuration is updated, it should check new values does not break the operation.
Example:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.