digest or user input as docker_push tag

[bazelbuild]

digest or user input as docker_push tag

[ixdy]

Unfortunately --workspace_status_command is fairly undocumented. I wrote a bit in #54 (comment), so I'm not going to repeat myself again here.

Kubernetes uses the stamp attribute in docker_bundle to use the git describe version string as the tag.
It's a somewhat complicated example, but how we have things set up:

1. We created a .bazelrc in our repo which runs --workspace_status_command hack/print-workspace-status.sh on all builds.
2. hack/print-workspace-status.sh calls into some other scripts which munge the git describe into a semantic version, and then we print these values, which will be saved by bazel into the stable-status.txt and volatile-status.txt files. Of note for this example is

STABLE_DOCKER_TAG ${KUBE_GIT_VERSION/+/_}

which adds a line like

STABLE_DOCKER_TAG v1.8.0-alpha.1.187_c34ee60f80acc7-dirty

into the bazel-out/stable-status.txt file.

3. Finally, in the docker_bundle rule we can use {STABLE_DOCKER_TAG} for the docker tag:

docker_bundle(
    name = binary,
    images = {"gcr.io/google_containers/%s:{STABLE_DOCKER_TAG}" % binary: binary + "-internal"},
    stamp = True,
)

(See my comment in the other issue as to why this is STABLE_DOCKER_TAG instead of DOCKER_TAG - basically, we want changes to this value to trigger rebuilds of the image.)

[mattmoor]

@sebgoa I'm going to open another issue to discuss this. @dlorenc and I have been thinking about this exact issue a fair amount lately and there is potentially room for collaboration.

[mattmoor]

@jmhodges Great question. Technically, this google/containerregistry library can precompute the digest and push-by-digest, but we don't currently expose this option. Frankly, I'm not 100% certain registries other than GCR would even accept this, but why I think that is a really long story :)

You may be interesting in the stamp = True option I just added to docker_push. @ixdy may have some tricks on getting interesting values into the status files this consumes.

On a semi-related note, @dlorenc and I have been debating what some useful rules_k8s might look like. Frankly, I am a die-hard advocate that K8s shouldn't be deploying using "immutable" tags and should be resolving tags to digests at a single cut-point of the deployment process. Among the rules I've wanted to experiment with is one that would allow you to bazel run :pod to build dependent containers, publish them, and deploy a pod from a checked in yaml (substituting certain tags for published digests). This is still 100% vaporware, but I'd love to get your thoughts on directions you might find useful.

[jmhodges]

Thanks for the response!

I have some open source code that I push to Docker Hub, so support for non-gcr would be helpful.

I don't quite follow what stamp does in the push context. I lost the thread at what --stamp-info-file does when passed to the pusher binary (which I can't seem to find a reference to on the web).

I would love a rules_k8s rather than building my own! The k8s API doesn't seem to support using digests instead of tags, so I'm pretty focused on getting the tagging right. Being able to change the image of a deployment's container spec from a bazel target only when its dependencies change is exactly what I'd like.

[jmhodges]

(Also I might just not understand the whole of what stamping means in the bazel context, in general. The docs are a little vague about what you can do with it.)

[mattmoor]

@jmhodges You should be able to pass a digest anywhere you can pass a tag reference to K8s.

re: DockerHub
All of this should support DockerHub, but we make you specify the domain they use as-if they were any other registry. See the docs, but registry = "index.docker.io" is what you want.

re: stamping
Basically, if you pass stamp = True to docker_push or docker_bundle you can pass variables to name components and we will substitute some values for them. If you add the option and look under bazel-out I think you will see stable-status.txt and volatile-status.txt, which should give you an indication of the variables available (I believe you can augment this, but @ixdy has a bit more experience with this than me). I realized yesterday that this didn't make it into the README.md when I recently added it to docker_push, so I'll open a separate issue to track this. Not the best, but see here for an example.

[jmhodges]

Ah, cool, so it's for including data that's globally available, but not for specific targets. Thanks and too bad!

[mattmoor]

@jmhodges I wanted to follow up and see whether there was still anything actionable left here that you wanted to see us implement? I'm about to look at improving the stamping documentation in the README.

[sebgoa]

@mattmoor don't want to hijack the issue, but picking up on your k8s rules idea above. What I am trying to do is make a build within a Pod and push to a private registry. So that I can run my container builds as k8s jobs.

I might open another issue, but roughly speaking the assumption is that I don't have bazel or docker locally, the builds are triggered by a k8s controller when I create a TPR object. It is for creating custom runtime for http://kubeless.io/.

[mattmoor]

(to be clear, not the TPR aspect, but the runtime bit)

[mattmoor]

I think everything from this thread is covered by the other issues, so closing.

[jmhodges]

Ugh, sorry, I keep meaning to come back here and mention that I had to fork docker_push in order get it to read the image tag from a file so I could do the shasum work using bazel's APIs.

I'm doing images the k8s-approved way and it simplifies a lot of re-building-and-deploying-every-commit problems without a lot more CI infra. Okay, have a good day!