rafecolton/docker-builder

Docker builder builds Docker images from a friendly config file.

docker-builder's Issues

panic when parsing `git:` tag from non git directory

If we try to run builder from somewhere else, bad things happen:

• if that somewhere else is another git repo, we get the wrong git info (branch name / rev)
• if that somewhere is not a get repo, we panic on line 93 of parser/tag/tag.go

What should we do instead? Since we're shelling out to git, we should probably use the Bobfile's directory as top instead of $PWD

post build hook for running black box tests

maybe with bats, since it's pretty universal

Add option to skip push for a given container build

If multiple containers are being built as tags of the same repo, they probably only need to be pushed once.

make repo dir configurable

Easy Travis and other CI webhook integration.

It would be great if I could run the HTTP server and specify a Bobfile for a path. Then I could give that path to Travis webhooks and have it hit the build server after successful CI.

Something like http://localhost:5000/docker-build/project/travis, perhaps?

specify work dir

$BOB_WORK_DIR

dockercreds

$BOB_DOCKER_CREDS

Add versioning to the bob.toml files

Make it a required field and keep around old struct versions to make sure everything is in sync

Rearrange so package main is at the top level

And move package builder to the builder/ directory

An "init" command to write an example Bobfile

Run individual push commands for each tag as opposed to pushing the repo as a whole

Clean up the webhook package `testMode` thingy

Add /health endpoint

I want to put docker-builder behind an AWS EC2 Elastic Load Balancer, but it requires a health check to make sure the instance is in service. Let's add a route at /health that always returns 200.

Bob panics when an invalid DOCKER_HOST is set

exits 2, which is ok - should at least handle the error gracefully if possible and not panic

Accept top-level ".bobconfig" for repo globals

Since the user has limited control over the data sent with a Travis/GitHub webhook, certain options will have to be specified statically in the repo.

The suggested implementation for this is a .bobconfig file at the top level of the repo. Example options to be included in the .bobconfig are the default Bobfile and container group to be built (ala #96)

Re-add hipchat/campfire notifications

There are a couple of things to consider:

• Do the tokens get provided on the cl / in the env for the entire server (probably the MVP)? If so, what if we want to use different accounts for different builds?
• Should the hipchat info be provided in the Bobfile? If so, how do we deal with private information (i.e. api tokens) in the Bobfile of a public repo? Something like travis secure env vars is possible, but not ideal given that there is no centralized docker-builder service.
• If we add this functionality, we probably don't want the notifications going off when we build locally (or at least an option to turn them off).
  • Should they be off or on by default?

Make sync build wait for `docker push`

It would be nice if this waited until after the push completed in case you want to block until you are able to pull the container for testing or deployment.

add option retries for failed `docker push`

delete temporary uuid tag after processing [container] section

Write document on mini-vm model vs process model for containers

and various other container-building strategies

HTTP endpoint for viewing logs

Make sure log level can be set externally if used as a library.

Also, container output seems a bit verbose - should it only be printed when log level is set to "debug" and not "info"?

option for bob to get creds from env or from .dockercfg (default) or any other file or script

Mysterious intermittent test failures

The tests that copy files from the repo dir to the workdir occasionally fail

Do we have a race condition?

Possible causes:

• set library
• File-copying

Project is no longer 'go get'-able

A go get of github.com/modcloth/docker-builder tries to use master rather than the travis-auth branch. I've worked around this by just using the binary release for the interim. I'm not sure if the correct course is to just update the README to direct users to the releases.

Add output indicating how long each container build/tag/push took

Maybe totals as well as separate times for builds and pushes?

Allow Synchronous HTTP Requests

Ala https://github.com/modcloth/docker-builder/blob/webhook-refactor/webhook/webhook.go#L61

Occasional Failure of Parser Tests

Due to http://golang.org/doc/go1.3#map (in Go 1.3, maps are iterated over in a random order).

This can be fixed by using a different data structure in the Bobfile, but that will require a significant refactor as well as the implementation of versioning for the Bobfiles (per #8)

Fix has been started in the bobfile-version-1 branch

Rename

Maybe "docker-builder" - otherwise this will never get discovered. Plus I'm still not sold on the name.

Providing GitHub API token if building with webhook?

If no API token is provided to the build server itself, and the request comes through on a webhook as opposed to a request to /docker-build, it wouldn't be possible to clone down a private repo. Do we care? If so, proposed solutions?

add `work` command - integrate worker?

Clean up conditional in builder Build() loop

Specifically, find a way to remove this code: https://github.com/modcloth/docker-builder/blob/3d7b23b0073a9dc423011d011c5950ababb7f2b1/builder/bob.go#L157-L162

(And then the Type() function can also be removed from the DockerCmd interface)

Make bob configurable through the environment

Allow Injection of ENV variables

into the Dockerfile. Allow them to be retrieved from the docker-builder process env

Builds aren't isolated

Builds on the same build server aren't isolated from each other. This could definitely be a problem if we build a bunk version of a container upon which another container depends. Addressing this will be quite involved and may involve a re-architect of the whole build server system.

Welcoming suggestions on the design for this.

Rename Builderfile

Empty response from server

Some requests get an error when posted via curl:

curl -XPOST -H Content-Type: application/json -u USERNAME:PASSWORD --connect-timeout 10 --silent --show-error https://bob.core-os.net/docker-build -d '
{
  "account": "USERNAME",
  "repo": "PROJECT",
  "ref": "27e5c49b64c846dc5d814945a16dd49384e022bb",
  "api_token": "APITOKEN",
  "sync": true
}
'
curl: (52) Empty reply from server

Will dig into this.

Make lint accept a list of files instead of just one file

Would probably make it easier to use with tests

Use Docker's tailfile package instead of os/exec for tailing logs

https://github.com/dotcloud/docker/tree/master/pkg/tailfile

Ref: http://crosbymichael.com/the-lost-packages-of-docker.html

Specify Bobfile path or a [[container]] name to build in REST Interface

In my setting, I have many different services under the same git repo. I need to build like 5-6 different images and i'd like to not have to build all of them everytime I push to github or perhaps even when I manually queue a build via the REST interface.

It would be very helpful adding a [[container]] name or a bobfile path when doing something like the following:

curl -s -XPOST http://localhost:5000/jobs -d '{"account":"modcloth","repo":"docker-builder","ref":"master","bobfile":"services/backend-service-1/Bobfile"}'

or maybe:

curl -s -XPOST http://localhost:5000/jobs -d '{"account":"modcloth","repo":"docker-builder","ref":"master","container_name":"backend-service-1"}'

cc @rafecolton

Server stops pushing after a while

I believe this to be caused by an issue within queued-command-runner. Will be addressed with a current pull request that, among other things, removes the queued-command-runner from use. Will link to the issue shortly.

Make public Dockerfile

Right now it isn't possible to just do a docker build . since the quay.io registry is private. It would be nice to have a Dockerfile.public or something similar to push up to docker.io (modcloth/docker-buiilder) or allow users to easily build a container.

`tag_opts` stanza does not work correctly

Looks like bob includes "<IMG>" in the command instead of the tag_opts

Add `BuildFromFile()` Function

Add something like bob.BuildFromFile() so it is no longer necessary to explicitly call Parse() before calling Build()

Add `--skip-push` cli option for build

Something is junking up the logs

The issue detailed in the description here is still happening.

Remove as many exec.Cmd instances as possible (ongoing)

The executable is `docker-builder` when I `go get`

and I want to be able to use go get. 😺 👏

handle providing dockercfg

Figure out why bats tests don't display correctly on Travis

Verify that a failed bats test will still correctly mark the travis run as failed.

Silence Martini output for /health

It's annoying to dig through all of the logs that are just GET requests for /health. Let's silence the output here.