evertrue/exhibitor-cookbook

Cookbook for Netflix Exhibitor

Exhibitor

Chef cookbook for installing and managing Netflix's Exhibitor, a co-process for Apache Zookeeper.

Caveats

Attempting to install Exhibitor, using Maven, on Ubuntu 14.04, will result in an error:

[ERROR] Plugin org.apache.maven.plugins:maven-clean-plugin:2.5 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-clean-plugin:jar:2.5: Could not transfer artifact org.apache.maven.plugins:maven-clean-plugin:pom:2.5 from/to central (https://repo.maven.apache.org/maven2): java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty -> [Help 1]

As a result, we do not support this.

Usage

In particular, two key attribute hashes drive this cookbook.

The first is node['exhibitor']['cli'], which specifies command-line options that will be used when Exhibitor is run. Some are necessary, and in particular the defaults provided will ensure Exhibitor is able to run.

The second is node['exhibitor']['config'], which manages the configuration parameters that get rendered to the exhibitor.properties file. The defaults are sensible starting values.

We recommend running exhibitor::default to get a basic, default setup of Exhibitor going, as well as calling exhibitor::service if you want the service to boot up in the same run. These recipes are split for workflows within, for example, AMI pipelines.

More documentation to come. Please see the Exhibitor docs for more information on the specifics of running Exhibitor.

Author and License

EverTrue [email protected]
Simple Finance [email protected]

Apache License, Version 2.0

exhibitor-cookbook's Issues

Please submit to supermarket.chef.io

Using directly from the repository is a bit clunky and it would be great if the maintainers could create a release and push to supermarket.chef.io

Error executing action `start` on resource 'runit_service[exhibitor]'

I am running into this running centOS 6.5. Although it looks like everything installsand starts up OK - but it would be nice if the cookbook didn't fail.

10.6.20.87 Error executing action start on resource 'runit_service[exhibitor]'
10.6.20.87 ================================================================================
10.6.20.87
10.6.20.87 Mixlib::ShellOut::ShellCommandFailed
10.6.20.87 ------------------------------------
10.6.20.87 Expected process to exit with [0], but received '1'
10.6.20.87 ---- Begin output of /sbin/sv start /etc/service/exhibitor ----
10.6.20.87 STDOUT: warning: /etc/service/exhibitor: unable to open supervise/ok: file does not exist
10.6.20.87 STDERR:
10.6.20.87 ---- End output of /sbin/sv start /etc/service/exhibitor ----
10.6.20.87 Ran /sbin/sv start /etc/service/exhibitor returned 1

Possibly related to this:

chef-cookbooks/runit#142

If so let's close this, but I wanted post incase they are not related (still learning chef).

Refactor to take advantage of zookeeper ~> 8.0

As of zookeeper cookbook v7.0.0, that cookbook is a true library cookbook, entirely made up of custom resources.

This cookbook should be updated to take advantage.

Spec

• Make effective use of the new zookeeper resources (available >= 7.0.0)
  • This has the potential to loosen the coupling of exhibitor to zookeeper

Can't run apt-get update before installation of chef-exhibitor

If you attempt to install chef-exhibitor, it needs to install 'patch'.

Because the cookbook uses end.run_action(:install) on the 'patch' installation resource, this makes it so that the patch resource is installed before anything else.

This causes a big problem if you haven't run apt-get update yet. Adding the 'apt' resource to your run list won't work around the issue because the end.run_action(:install) short circuits the run list completely, making it impossible to run apt first.

Additional information on why end.run_action(:install) causes problems.

https://www.chef.io/blog/2013/09/04/demystifying-common-idioms-in-chef-recipes/
https://gist.github.com/spuder/42e9b94bf557520458fa#file-output-txt-L279

Unless there is a real need for this, I think it should be taken out and the 'patch' resource should just be installed during the normal execution run.

I haven't been able to find a work around to this issue besides ssh-ing into each node one by one and running apt-get update before provisioning.

The default transaction and snapshot dir /tmp/zookeeper are removed after reboot

When you use the default configuration zookeeper does not come up after reboot again because the
transaction and snapshot dir is lost. Shouldn't the startup script check if they exist and create them when they are missing?

Correctly install Exhibitor via download method

As discovered while testing #34, using node['exhibitor']['install_method'] = 'download' does not work properly, as the standalone JAR is a library JAR, not a runnable one (see soabase/exhibitor#284 for details).

Transfer repo ownership to EverTrue

@jakedavis seeing as we’ve taken over the zookeeper cookbook, I was wondering if you’d like to transfer this one to us, as well? We already own the Supermarket releases.

Beef up README docs

The README here is pretty sparse with regards to how best to use this cookbook, and how it integrates with the zookeeper cookbook. Came up during discussion of evertrue/zookeeper-cookbook#155.

That should be remedied.

Example S3 attributes are incorrect

The example S3 attributes:

set['exhibitor']['s3']['key']
set['exhibitor']['s3']['secret']

are incorrect, based on how they're rendered. Based on the Exhibitor docs, they should be:

set['exhibitor']['s3']['access-key-id']
set['exhibitor']['s3']['access-secret-key']

Config template (re)render should restart exhibitor

It doesn't look like the exhibitor service gets restarted on template change (such as adding new zk node)

Default Apache mirror for Zookeeper doesn't store previous versions

I believe this cookbook should not throw exceptions if the required version of Zookeeper wasn't found in the stable mirror. Instead it should search for it in the archive: https://archive.apache.org/dist/zookeeper/

================================================================================
Error executing action `install` on resource 'ark[zookeeper]'
================================================================================

Net::HTTPServerException
------------------------
remote_file[/var/chef/cache/zookeeper-3.4.10.tar.gz] (/var/chef/cache/cookbooks/ark/providers/default.rb line 41) had an error: Net::HTTPServerException: 404 "Not Found"

Cookbook Trace:
---------------
...

Resource Declaration:
---------------------
# In /var/chef/cache/cookbooks/zookeeper/resources/default.rb

 69:   ark 'zookeeper' do
 70:     url         "#{mirror}/zookeeper-#{new_resource.version}/zookeeper-#{new_resource.version}.tar.gz"
 71:     version     new_resource.version
 72:     prefix_root install_dir
 73:     prefix_home install_dir
 74:     checksum    new_resource.checksum if new_resource.checksum
 75:     owner       username
 76:     group       username
 77:   end
 78:

Compiled Resource:
------------------
# Declared in /var/chef/cache/cookbooks/zookeeper/resources/default.rb:69:in `block in class_from_file'

ark("zookeeper") do
  action [:install]
  retries 0
  retry_delay 2
  default_guard_interpreter :default
  declared_type :ark
  cookbook_name "exhibitor"
  url "http://apache.mirrors.tds.net/zookeeper//zookeeper-3.4.10/zookeeper-3.4.10.tar.gz"
  version "3.4.10"
  prefix_home "/opt"
  prefix_root "/opt"
  owner "zookeeper"
  group "zookeeper"
  extension "tar.gz"
  prefix_bin "/usr/local/bin"
  home_dir "/opt/zookeeper"
  path "/opt/zookeeper-3.4.10"
  release_file "/var/chef/cache/zookeeper-3.4.10.tar.gz"
end

Platform:
---------
x86_64-linux

Allow mirroring zookeeper

Currently the cookbook installs zookeeper 3.4.9

The zookeeper repos don't keep old versions. This can break when new versions of zookeeper come out.

Its best to mirror zookeeper, which this cookbook doesn't appear to support.

zk cookbook update?

is there anything blocking bumping the zk cookbook to version 6? I think version 7 has some breaking changes, but afaict 6 looks pretty safe

Issues building the exhibitor immage

I am running into issues with the cookbook. it gives me a "Could not satisfy version constraints for: zookeeper". I made sure i had a working zookeeper and all the other dependencies that go with the build of exhibitor. Kindly help!!!..
My zookeeper build came from your repo, and that was successful.
Thanks.

zookeeper (7.1.0)
maven ~> 2.2
et_gradle ~> 2.0
magic ~> 1.5
runit ~> 1.7
java ~> 1.35
build-essential >= 0.0.0

configtype file is broken because some properties still use underscore

When replacing render_properties_file with properties_config a few properties were missed (lines 85-88 in recipes/default.rb) which contain the zookeeper directories.

Those specific properties are needed in order for Zookeeper to start up, so Exhibitor sits in latent mode.

Basically all that's needed is to replace underscores with hyphens and generate a new release. As it stands right now, the cookbook is non-functional for file based configs.

Custom resource

It would be nice if this cookbook was converted to use a custom resource so it is more consistent with the zookeeper cookbook

Add tests

Could use a few.

Couple slightly more to Zookeeper

One thing I've noticed while deploying this and Zookeeper is they don't think about each other very well. That is, you'll render the Zookeeper config, render Exhibitor config using different attributes, and then Exhibitor will run over the Zookeeper config. Exhibitor should probably lean more on Zookeeper attributes so this is less problematic and you'll know what the resultant config will be.

Zookeeper installation does not honor install_dir attribute

My organization has a need to install Exhibitor and Zookeeper in a directory other than /opt/. While most of the facilities to do this are in both the Zookeeper and Exhibitor cookbooks, it appears that the Exhibitor cookbook is not correctly passing the custom installation path to the Zookeeper lwrp to install the software in default.rb. For now I have patched this locally and gotten it to work by including the install_dir directive in my local copy used to build our servers. Because of a legal NDA I signed within my organization, I am unable to submit pull requests to open source projects.

The solve for this is to change the library call to the following:

zookeeper node['exhibitor']['zookeeper_version'] do
  username  node['exhibitor']['user']
  user_home "/home/#{node['exhibitor']['user']}"
  install_dir node['zookeeper']['install_dir']
end

Thank you!

Refactor

Want to rewrite this as we are in evertrue/zookeeper-cookbook#83, as an LWRP.

It carries with it several design decisions that should make this much nicer.

gradle command not found and permission denied on CIS hardened systems

If /usr/local/bin is not part of PATH, the gradle call will fail.

since line 43 of exhibitor::gradle hardcodes '/usr/local/bin/gradle', I believe we should explicitly set it in the exhibitor::_exhibitor_build command also, so we do not need to additionally modify $PATH when necessary.

specifically using sh to run the command avoids some permission issues on hardened systems, as well (CIS, for instance).

dependency is out of date

The zookeeper cookbook was bumped to 3.0.0, however the metadata in this cookbook is still referencing ~> 2.0

This means that I can't have my wrapper cookbook point to 3.0.0 if I want it to install both zookeeper and exhibitor

Chef inadvertently manages shared config instead of default

Running in file mode with my options specified in the node at normal level:

   "exhibitor": {
      "cli": {
        "fsconfigdir": "/mnt/exhibitor_state/"
       }
    },

Chef keeps managing the shared config file:

Recipe: exhibitor::default
  * directory[/opt/exhibitor] action create (up to date)
  * directory[/tmp/zookeeper] action create (up to date)
  * directory[/tmp/zookeeper_log_indexes] action create (up to date)
  * file[/mnt/exhibitor_state/exhibitor.properties] action create
    - update content in file /mnt/exhibitor_state/exhibitor.properties from 75c634 to f6cfc4
    --- /mnt/exhibitor_state/exhibitor.properties   2015-06-29 12:47:42.339391000 -0700
    +++ /mnt/exhibitor_state/.exhibitor.properties20150629-29319-7qzuj3 2015-06-29 12:48:48.461299000 -0700
    @@ -1,49 +1,15 @@

Comments at https://github.com/SimpleFinance/chef-exhibitor/blob/master/attributes/default.rb#L34 mention the hash should be managing the default config, but the recipe manages the shared config in https://github.com/SimpleFinance/chef-exhibitor/blob/master/recipes/default.rb#L71-L74:

  node.default[:exhibitor][:cli][:fsconfigdir] = '/tmp'
  node.default[:exhibitor][:cli][:fsconfigname] = 'exhibitor.properties'

  file ::File.join(node[:exhibitor][:cli][:fsconfigdir], node[:exhibitor][:cli][:fsconfigname]) do

I'm not sure of the purpose of the first 2 lines because they won't get used since I'm applying my config at the normal level. However I think the file resource should be node[:exhibitor][:cli][:defaultconfig] instead of what's there.