CI_INFRASTRUCTURE_CF

Provisions a jenkins machine on the cloud with a set of pre configured jobs that deploy Microbosh, Bosh and Cloudfoundry on demand.

Motivation

As BOSH operators, we were facing repetitive manual processes to deploy our infrastructure in different environments and regions. We were also setting the same attributes for different releases realizing that our BOSH deployment manifests at common configurations. Another issue we were facing was the lack of standardized deployment procedures between different members on the team.

Goals

Automation for BOSH deployments (including BOSH and Cloud Foundry out of the box)
Reuse configurations between deployments (eg: net_ids, network_ranges, etc)
Keep full infrastructure configuration in a single place (provision via Vagrantfile)
Automated updates and maintenance for BOSH deployments

Disclaimer

This project together with this documents aims to show the goals and current state of the tool. The tool will require additional development before we can release a stable version; however, we recommend trying it on a development environment. We are unable to give support to the community at this time but hope to do so in the near future.

Technologies

Chef
Vagrant

Plataform support

Openstack

Local pre deployment setup

###On Linux(Ubuntu 14.04) Install dependencies:

  sudo apt-get update
  sudo apt-get install linux-headers-$(uname -r)
  sudo apt-get install git
  wget https://dl.bintray.com/mitchellh/vagrant/vagrant_1.6.3_x86_64.deb
  sudo dpkg -i vagrant_1.6.3_x86_64.deb
  sudo apt-get install virtualbox
  # ONLY FOR 12.04 =============
  sudo apt-get install python-software-properties 
  sudo add-apt-repository cloud-archive:icehouse
  sudo apt-get update
  sudo apt-get dist-upgrade
  # ============================
  sudo apt-get install python-novaclient  #pending to test
  wget https://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/12.04/x86_64/chefdk_0.2.0-2_amd64.deb
  sudo dpkg -i chefdk_0.2.0-2_amd64.deb
  # Installs Quantum ===========
  sudo apt-get install language-pack-en
  sudo apt-get install python-quantumclient
  echo 'export LC_ALL=en_US.UTF-8' >> ~/.bashrc
  # ============================

###On OSX Install dependencies:

TODO

###On Both:

Configure nova client:

  $ export OS_USERNAME=user
  $ export OS_PASSWORD=password
  $ export OS_TENANT_NAME=tenant
  $ export OS_AUTH_URL=https://example.keystone.com:5000/v2.0

Install Vagrant plugins:

  $ vagrant plugin install vagrant-berkshelf
  $ vagrant plugin install vagrant-openstack-plugin
  $ vagrant plugin install vagrant-omnibus

Cloud pre deployment setup

###On openstack

Create 2 networks
- Internal (CF traffic)
- External (CF >> Internet traffic)

Create keypair for vagrant

  $ nova keypair-add vagrant > ~/.ssh/vagrant.pem

Create Jenkins and SSH sec-groups

  $ nova secgroup-create jenkins "Jenkins sec group"
  $ nova secgroup-add-rule jenkins tcp 8080 8080 0.0.0.0/0
  $ nova secgroup-create ssh "SSH sec group"
  $ nova secgroup-add-rule ssh tcp 22 22 0.0.0.0/0

Provision floating IP for Jenkins and for CF:

  $ nova floating-ip-create external #To be used for Jenkins
  +--------------+-----------+----------+----------+
  | Ip           | Server Id | Fixed Ip | Pool     |
  +--------------+-----------+----------+----------+
  | 1.1.1.2      |           | -        | external |
  +--------------+-----------+----------+----------+
  $ nova floating-ip-create external #To be used for CF
  +--------------+-----------+----------+----------+
  | Ip           | Server Id | Fixed Ip | Pool     |
  +--------------+-----------+----------+----------+
  | 1.1.1.3      |           | -        | external |
  +--------------+-----------+----------+----------+

##Attributes

node[:ci_infrastructure_cf][:jobs] contains hashes were the keys are the jobname and the values are theirs configurations.

###For Microbosh:

See complete list of attributes at attributes/microbosh.rb.

####Required:

node[:ci_infrastructure_cf][:jobs][:microbosh][:provider][:name] can be openstack|aws|vsphere. Default: openstack.
node[:ci_infrastructure_cf][:jobs][:microbosh][:provider][:user] provider username. Default: admin.
node[:ci_infrastructure_cf][:jobs][:microbosh][:provider][:pass] provider password. Default: admin.
node[:ci_infrastructure_cf][:jobs][:microbosh][:provider][:tenant] provider tenant. Default: dev.
node[:ci_infrastructure_cf][:jobs][:microbosh][:provider][:auth_url] keystone url. Default: https://example.com:5000/v2.0/tokens.
node[:ci_infrastructure_cf][:jobs][:microbosh][:provider][:subnet_id] Internal subnet id. Default: SUBNET_ID.

##Usage

Clone Repo:

  git clone https://github.com/cloudfoundry-community/ci_infrastructure_cf.git
  cd ci_infrastructure_cf
  bundle install

Export environment variables required on the vagrantfile:

  # env_vars file
  export OS_USERNAME=admin
  export OS_PASSWORD=admin
  export OS_FLAVOR=m1.large
  export OS_IMAGE=ubuntu-14.04
  export OS_AUTH_URL=https://keystone.example:5001/v2.0
  export OS_KEYPAIR_NAME=vagrant
  export OS_NETWORK=internal
  export OS_TENANT_NAME=development
  export JENKINS_FLOATING_IP=1.1.1.2
  export MICROBOSH_SUBNET_ID=53e020ad-bc34-4126-be44-e0a3e2c04591
  export MICROBOSH_IP=FIXED_INTERNAL_IP

####Deploy:

  $ vagrant up --provider=openstack

Update configuration on the Vagrantfile.

Re-Provision VM

  $ cic provision

Re-Run tasks manually

Go to http://FIXED_JENKINS_IP:8080 :

Run any task manually:

##Command Line tool

On the bin folder you will find a command line application. It will let you generate initial stubs for CF and Bosh. It will also let you edit your configuration prior to provision of the jenkins machine.

###CF|Bosh initial setup and customizations.

A wizard will let you generate intial configurations for bosh or cf:

  $ cic generate_stub <cf|bosh>

You can also edit the generated file with:

  $ cic edit_stub  <cf|bosh>

Now you can reprovision this configuration on the Jenkins machine so that they can take effect:

  $ cic provision

Troubleshooting on openstack

Security groups quota limit exceeded:

if you get the following errror when running any of the tasks:

  "409-{u'NeutronError': {u'message': u\\\"Quota exceeded for resources: ['security_group']\\\""}}"

You can try by changing the quota limts using admin credentials with the following command:

  $ neutron quota-update --tenant-id b5e6943e8280489wb86c4943a6a317ab --security-group 1000 --security-group-rule 100000

VM creation failed:

message:

 Bosh::Clouds::VMCreationFailed (Bosh::Clouds::VMCreationFailed)

Possible causes:

IP of one of the vms already taken.
Not enough permissions on the openstack user.

cloudfoundry-community / ci_infrastructure_cf Goto Github PK