Giter Site home page Giter Site logo

cd-demo's Introduction

cd-demo

A continuous delivery demo using Jenkins on DC/OS.

This demo is a Python script that, when run with the install command, will:

  1. Installs Jenkins if it isn't already available.

Running the pipeline command will:

  1. Sets up a Jenkins "workflow" pipeline job and the necessary credentials to demonstrate a basic continuous delivery pipeline. Jenkins will:

    • Spin up a new Jenkins agent using the Mesos plugin. This agent runs inside a Docker container on one of our DC/OS agents.
    • Clone the git repository
    • Build a Docker container based off the Jekyll Docker image that includes the content stored in /site and push it to DockerHub.
    • Run the newly created container and a Linkchecker container that runs a basic integration test against the container, checking that the webserver comes up correctly and that all links being served are valid (i.e. no 404s).
    • Manually trigger a Marathon deployment of the newly created container to the DC/OS base Marathon instance. If the application already exists, Marathon will simply upgrade it.
    • Make the application available on a public agent at port 80 using Marathon-lb.

When run with the dynamic-agents command, it will:

  1. Creates 50 build jobs that take a random amount of time between 1 and 2 minutes. These jobs will randomly fail.
    • The Mesos plugin will spin up build agents on demand for these jobs, using as much capacity as your cluster has available.
    • When these jobs are finished, the Jenkins tasks will terminate and the resources will be relinquished back to other users of your cluster.

When run with the uninstall command, it will:

  1. Remove any persisted credentials, build job and view configurations.
  2. Uninstall Jenkins.

bin/demo.py --help will show you full help text and usage information.

Basic Usage

Prerequisites

  • Python 3
  • pip install -r requirements.txt

Set Up

  1. Clone this repository!

    git clone https://github.com/mesosphere/cd-demo.git
    
  2. Create a branch from the latest stable tag, this is mandatory:

    git checkout v1.8.6-2.7.2
    git checkout -b my-demo-branch
    git push origin my-demo-branch
    
  3. Ensure you have a DC/OS cluster available. 1 node will work but more than 1 node is preferable to demonstrate build parallelism.

  4. Export the demo Docker Hub password (NOT the DC/OS cluster password) to an environment variable (ideally put it in your ~/.bashrc). You will need to replace the password here with the password for the cddemo user with permission to push to mesosphere/cd-demo-app (or your own repo, if you override the --org and --username flags later):

    export PASSWORD=mypass123
    

Running Demo

  1. Run the install command. This is mainly a wrapper for the dcos package install command but will also check to see if you're authenticated.

    python3 bin/demo.py install http://my.dcos.cluster/
    

    NOTE: You must use the domain name for your cluster; the IP address will fail.

  2. You can now run either the pipeline demo or the dynamic agents demo. To run the pipeline demo, you will also need to specify the ELB address (Public Agent):

    python3 bin/demo.py pipeline  --password=$PASSWORD http://my.elb/ http://my.dcos.cluster/
    
  3. The script will first install Marathon-lb if it looks like it isn't available. It will also update the marathon.json in the branch you specified to include the ELB hostname so that Marathon-lb can route to it.

  4. The script will then use the Jenkins HTTP API to install jobs and necessary credentials. It will automatically trigger the initial build before finishing.

  5. Navigate to the Jenkins UI to see the builds in progress. After a few seconds, you should see a build executor spinning up on Mesos. If you navigate to the job, you'll see the pipeline in progress.

  6. The deploy will happen almost instantaneously. After a few seconds, you should be able to load the application by navigating to the ELB hostname you provided earlier in your browser.

deployed-app

  1. Now let's run the dynamic agents demo. It will create 50 jobs that will randomly fail.

    python3 bin/demo.py dynamic-agents http://my.dcos.cluster/
    
  2. Navigate back to the Jenkins and/or DC/OS UI to show build agents spinning up manually.

Uninstalling

  1. Simply run the uninstall command to remove any persisted configuration and to uninstall the DC/OS service itself. This will allow you to run multiple demos on the same cluster but you should recycle clusters if the version of the Jenkins package has changed (to ensure plugins are upgraded):

    python3 bin/demo.py uninstall http://my.dcos.cluster/
    

Alternatively, run the cleanup command instead to just remove jobs and to avoid having to re-install Jenkins.

Advanced Usage

Using a Custom Docker Hub Organisation

By default, this script assumes you will be pushing to the mesosphere/cd-demo-app repository on Docker Hub.

  1. Create a public repo called cd-demo-app under your example organisation.

  2. Create a Docker Hub user that has credentials to push to this repository.

  3. Run the pipeline demo, passing in the credentials:

    python3 bin/demo.py pipeline --org=myorg --username=myuser --password=$PASSWORD http://my.elb/ http://my.dcos.cluster/
    

Build on Commit

  1. Run the demo to completion. The pipeline will continue to monitor your branch after the script finishes:

    python3 bin/demo.py pipeline --password=$PASSWORD http://my.elb/ http://my.dcos.cluster/
    
  2. Create a new blog post with today's date, open it up in your text editor and make whatever changes you'd like to:

    cp site/_posts/2016-02-25-welcome-to-cd-demo.markdown site/_posts/$(date +%Y-%m-%d)-my-test-post.markdown
    nano site/_posts/$(date +%Y-%m-%d)-my-test-post.markdown
    
  3. Commit your changes and push them up to GitHub:

    git add site/_posts/$(date +%Y-%m-%d)-my-test-post.markdown
    git commit -m "Demo change"
    git push origin my-demo-branch
    
  4. Jenkins will pick up the change within a minute and kick off the pipeline. If you want to fail the build, simply insert a broken link into your post.

Demonstrating Multi-tenancy

To demonstrate how you can install multiple Jenkins instances side by side on DC/OS, simply give your Jenkins instances unique names using the --name argument and run the demo as follows. Note that if you only have one public agent, you will not be able to deploy applications from multiple pipelines (each application requires port 80).

  1. Create one instance:

    python3 bin/demo.py install --name=jenkins-1 http://my.dcos.cluster/
    
  2. Open a new terminal tab and create a second instance and so on:

    python3 bin/demo.py install --name=jenkins-2 http://my.dcos.cluster/
    
  3. You can uninstall these in the same way:

    python3 bin/demo.py uninstall --name=jenkins-1 http://my.dcos.cluster/
    python3 bin/demo.py uninstall --name=jenkins-2 http://my.dcos.cluster/
    

Authentication

The script will check to see if your current machine has a valid dcos_acs_token set. If it doesn't:

  1. it attempts to authenticate using the default username and password for an Enterprise DC/OS cluster. You can override these using the --dcos-username and --dcos-password arguments.

  2. if this fails, it will attempt to use the --dcos-oauth-token arguments to authenticate against an Open DC/OS cluster.

TODO

  • This script is currently untested on Windows.

cd-demo's People

Contributors

ssk2 avatar sschneid avatar colin-msphere avatar brndnmtthws avatar ianpward avatar keith-mcclellan avatar vishnu2kmohan avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.