chaffelson/whoville

An opinionated auto-deployer for the Hortonworks Platform

Whoville

Cloudy Hortonworks Platforms in under 60 minutes

A set of demos in a small Docker Image with Orchestration for AWS/GCP/Azure deployment from a simple Profile. This project is implemented directly in python rather than using a framework like Ansible because, when we started, the level of customisation and process wrangling for the new open source project integrations seemed easier in that manner. As the underlying platform matured the need for those invasive processes diminished and now this installation is much better executed in Ansible on a Terraform infrastructure, which is now the Cloudera standard. This repository and project is kept for historic purposes.

Requirements

Hosting Platform

You will need an AWS account with rights to deploy machines, assume roles, etc.
Please see the Cloudbreak Requirements for creating AWS Roles.

Docker Deployment

Everything is included if you use the provided Docker Image for development or deployment

Local Deployment

Required: Python3.6
Recommended: A Python IDE with inspection and Github support, such as Pycharm

Quickstart

1. Customise a profile.yml:

   curl -sSL http://bit.ly/WhovilleProfile > profile.yml && vi profile.yml

2. Install / Start a recent version of Docker

3. Export your profile path and run the Docker Image:

   export PROFILE=/path/to/my/profile.yml
   docker run -ti -v ${PROFILE}:/profile.yml:ro chaffelson/whoville:latest

• Make sure to mount any addition resource volumes specified in your Profile e.g. '-v /MyDemos/:/MyDemos/'
• If you want to use interactive Deployment please append '-i' to the 'docker run' command, or see the Detailed Guide below

More Information

Suggested setup for Interactive Development in a local Python environment.
Detailed Definitions Guide for creating more demos and services.
Troubleshooting guide for known issues.

Support

This software is supplied as-is with no support guarantee under the Apache 2.0 license
Please raise any Issues on Github

You can link to this page from http://bit.ly/HwxQuickstart

whoville's Issues

ambari_wait status:

The deployment fails at this stage:

`Waiting for 90s before deploying cluster with services: AMBARI_METRICS HDFS MAPREDUCE2 YARN ZOOKEEPER DRUID STREAMLINE NIFI KAFKA STORM REGISTRY HBASE PHOENIX ZEPPELIN SOLR
Deploying the cluster once 1 hosts are registered.

/tmp/ambari-bootstrap-master/deploy/deploy-recommended-cluster.bash: line 77: hosts_regd[*]: unbound variable
Now open your browser to http://217.70.211.17:8080 and login as admin/StrongPassword to observe the cluster install
Waiting for cluster to be installed...
Traceback (most recent call last):
File "", line 2, in
IndexError: list index out of range
Traceback (most recent call last):
File "", line 2, in
IndexError: list index out of range
ambari_wait status:
ambari_wait status:
ambari_wait status:
ambari_wait status:
^C
[root@hdp11 ~]#
`

add owner tag to cloudbreak instance

is it possible to add a owner tag to the cloudbreak instance that whoville spins up to stop it getting autokilled by the cloudera aws bot