rtyler/blimpy

A Vagrant-inspired tool for managing multiple machines in the cloud

Blimpy

About

Blimpy is a tool to help developers spin up and utilize machines "in the cloud."

Once a developer has a Blimpfile, they can execute a few simple commands to manage the newly created "fleet" in the specified cloud provider:

   % blimpy start
   [snip]
   >> excelsior ..... online at: ec2-50-112-3-57.us-west-2.compute.amazonaws.com..
   >> goodyear ..... online at: ec2-50-112-27-89.us-west-2.compute.amazonaws.com
   %

Once machines are online, they're easy to access by name with:

  % blimpy scp goodyear secrets.tar.gz
  % blimpy ssh goodyear

Then once you're finished working with the machines a simple blimpy destroy will terminate the machines.

---

The Blimpfile

Here's an example Blimpfile:

    Blimpy.fleet do |fleet|
        fleet.add(:aws) do |ship|
            ship.name = 'rails-app'
            ship.ports = [22, 80, 8080] # [Optional] Create a security group with these ports open
            ship.image_id = 'ami-4438b474' # [Optional] defaults to Ubuntu 12.04 64-bit
            ship.livery = Blimpy::Livery::CWD # [Optional]
            ship.group = 'Simple' # [Optional] The name of the desired Security Group
            ship.region = 'us-west-1' # [Optional] defaults to us-west-2
            ship.username = 'ubuntu' # [Optional] SSH username, defaults to "ubuntu" for AWS machines
            ship.flavor = 'm1.small' # [Optional] defaults to t1.micro
            ship.tags = {:mytag => 'somevalue'} # [Optional]
            ship.provision_on_start = false # [Optional] defaults to true
        end
    end

Supported Clouds

Currently Blimpy supports creating machines on:

• Amazon Web Services - using the :aws argument passed into fleet.add
• OpenStack - using the :openstack argument passed into fleet.add

---

What is Livery?

In aviation, livery is the insignia or "look" an aircraft typically has. For example, Alaskan Airlines has a distinctive "creepy mountain man" livery on every plane.

With Blimpy, "livery" is a similar concept, a means of describing the "look" of a specific machine in the cloud. Currently the concept is still on the drawing board, but if you would imagine a tarball containing a bootstrap.sh script and Chef cookbooks or Puppet manifests to provision the entirety of the machine from start-to-finish.

When the machine comes online, the specified livery would be downloaded from S3 (for example) and bootstrap.sh would be invoked as root.

blimpy's Issues

0.4.3 fails on Ruby 1.8.7

-> % blimpy init
/home/tyler/.rvm/gems/ree-1.8.7-2011.12@flexd/gems/blimpy-0.4.3/bin/blimpy:7: uninitialized constant YAML::ENGINE (NameError)
        from /home/tyler/.rvm/gems/ree-1.8.7-2011.12@flexd/bin/blimpy:19:in `load'
        from /home/tyler/.rvm/gems/ree-1.8.7-2011.12@flexd/bin/blimpy:19

undefined method in resume

Hi,

I'm using blimpy with Ruby 1.9 and when I run blimpy start after adding a new ship, I get this:

Resuming: ,/usr/local/lib/ruby/gems/1.9/gems/blimpy-0.4.3/lib/blimpy/fleet.rb:61:in block in resume': undefined methodresume' for nil:NilClass (NoMethodError)

Is this a ruby 1.9 issue or a more general bug?

Thanks,
Steve

Support bootstrapping a basic livery on the remote host

Not storing the BlimpyFleetId anywhere

We're tagging machines, but we're not storing it in ./.blimpy.d/.

We probably should, then we may be able to use that to help validate management parts

Need a .blimpignore file for the :cwd livery

Basically a file with a bunch of rsync ignores for things that don't need to be sent to the machine

I broke Box#destroy

Note to self

fail

`blimpy provision name` broken with Box refactor

Whoops

Add `blimpy provision` to re-bootstrap the livery

This naming scheme is getting silly

`blimpy list` should list the status of the VMs configured in the Blimpfile

It'd be useful to know what VMs will come online when the blimpy start command is invoked

Hide access to the raw server under Blimpy::Box

There's no need for the Fleet to have that level of access

Machines coming up in US East 1

Despite the setting theoretically being us-west-2

Should check for ~/.fog or FOG_RC on startup and error nicely

Startup blimps should have a ridiculously gratuitous animated ascii blimp

       _..--=--..._
    .-'            '-.  .-.
   /.'              '.\/  /
  |=-                -=| (
   \'.              .'/\  \
    '-.,_____ _____.-'  '-'
jgs      [_____]=8

Validation errors shouldn't print a gnarly stack trace

Blimpy should support OpenStack

This one's going to hurt.

A livery URL should fetch the data and bootstrap it

Need a way to pass options into the livery script

For puppet specifically, being able to overwrite the manifest file and the module path would be handy

Liveries need a means to ask for ports to be opened

specs fail without a proper ~/.fog file on the test machine

Currently failing in BuildHive due to this issue.

Should create security group(s) automagically

Having something like:

fleet.add do |ship|
  ship.open_ports [22, 80]
end

Instead of requiring a security group to be set up by default would be handy

Should generate a new keypair when standing up new hosts

To make the blimpy ssh command as seamless as possible. Whenever a blimpy start is called, and new hosts are created, we should generate a new key pair for the machines in ever region there are machines

RSpec tests should be given their own working directory

Bootstrapping a livery should happen "in parallel"

Instead of running one machine after the other, we should start the provisions in parallel

rsync will fail if the host-key cannot be verified

We're not ignoring host key validation on the rsync for a :cwd livery, which can break on openstack

The .blimp file should fully reconstitute the Blimpy::Box obbject

blimpy ssh <name> should log me into the machine

Over SSH naturally!

`blimpy sync` should rsync the current directory to `/blimpy` on the remote host(s)

I broke `blimpy ssh`

SMOOTH MOVE FERGUSON

Box#validate! should make sure the instance_id exists in the region

Fucking AWS, how does it work

Reformat the `blimpy start` output to not suck so bad

If username is root, :cwd livery shouldn't sudo

FreeBSD images default to root logins and don't have sudo installed by default.

Add support for configuring machine sizes in Blimpfiel

If rsync does not exist on the remote host, fall back to tarball + scp

On FreeBSD hosts they do not have rsync installed by default, rats.

Failing to load a valid Blimpfile should report a more useful error

Currently an Blimpy::InvalidBlimpFileError is raised :(

Validation of security groups isn't using the Blimp's region

Should generate a new keypair when standing up new hosts

To make the blimpy ssh command as seamless as possible. Whenever a blimpy start is called, and new hosts are created, we should generate a new key pair for the machines in ever region there are machines

Need a `blimpy package` to snapshot a running VM as a new image

`blimpy scp <filename> [dest]` could be handy

AWS boxes should validate that the security group has port 22 open

`blimpy provision` with no arguments should run on all hosts

Need a `blimpy status` command

`blimpy start` should write the internal DNS name to the .blimp flie

We'll need this in the "future" to verify that the machine we SSH into is the right machine

Should refactor Blimpy::Box to support different "cloud" providers

I can see it making sense to have something like this in your Blimpfile:

Blimpy.fleet do |f|
  f.add do |s|
    s.cloud :aws
    s.name 'amazon'
  end
  f.add do |s|
    s.cloud :openstack
    s.name 'private'
  end
end

The Invalid Blimpfile error should give more information

"The Blimpfile is invalid!" Fuck you, what's wrong with it?

Engine isn't a useful internal abstraction, all of that mess should be within the fleet

I think the Fleet object should become the manager coordinator of multiple ships, etc

Should verify that the livery is valid before starting the machine

Since symbols (:cwd) are deprecated, they should fail the Blimpfile at validation instead of inside of #bootstrap_livery

`blimpy start` should resume instances that were stopped

Should have a built in :puppet livery

The :puppet livery should bootstrap Puppet onto the machine and provide the same workspace copying that the :cwd currently does.

This will probably require some amount of thought on how to properly configure liveries

`blimpy init` doesn't create the right Blimpfile

Doesn't generate the right calls to invoke an AWS box, regression since #17

`blimpy stop` should stop instances