Write end-to-end tests for mybinder.org

[jupyterhub]

Write end-to-end tests for mybinder.org

[yuvipanda]

There's one now that tests (2) and (3) under tests/ in this repo. I'm just structuring them as pytest tests, which is great!

[minrk]

For 1. to get maximum realistic test, is it possible to delete the image for the test repo prior to requesting the build? Does the test env have sufficient permission to do that?

[yuvipanda]

@minrk yeah, technically it can do that - since we already have the service account key for gcr.io login in config/secret/common.yaml. I guess we could just use that and do a delete before building. But I'm unsure how we can make that idempotent tho - if two intsances of this test are running at the same time, they'll trample each other.

Another possibility is we have a github repository that we make a new commit to each time we do this. We push it into a new branch, do the build, verify it works, then remove the branch. This will be much more idempotent test that trying to whitebox our way into deleting the image. What do you think?

[minrk]

@yuvipanda a dedicated test repo that we push trivial commits to (e.g. date > file) sounds like the best option.

[yuvipanda]

@minrk I've created an org (https://github.com/binderhub-ci-repos) for this.