package-dockertested: skopeo version skew breaks pre-flight checks

[openshift-eng]

package-dockertested: skopeo version skew breaks pre-flight checks

[sosiouxme]

We can scrum about who'll take on the CI updates. /cc @brenton

@juanvallejo feel like getting to know our CI better? :-P

[stevekuznetsov]

@sosiouxme @rhcarvalho please triage this. Today, fixing this issue is a blocker for getting new versions of Docker into the CI stream, so if you feel like you will not have the bandwidth to tackle it this week we can just turn off the check.

[sosiouxme]

I can make a PR to disable this check for now to unblock testing with rhel7next docker.

Who has been driving getting rhel7next docker into use in CI? It seems like it would be way more efficient for them to update that to pull in rhel7next skopeo as well than for me to try to figure out how to do everything @stevekuznetsov described above. And it feels like a better owner for this problem - we're just trying to consume skopeo and don't care how it gets there.

[stevekuznetsov]

This job and the workflow above are the mechanism for getting the latest docker into our CI. We can't install from rhel7next* on the normal CI AMIs as any repo we install from we basically need to always remain on for the future, and we cannot safely rely on the latest Extras compose. This is why the package-dockertested job exists to take the bleeding edge docker, test it, and when it's safe, tag it into the tested repository for consumption by the normal CI AMIs.

[sdodson]

Thanks, now I understand why we can't just leave the repo enabled in its current form.

What if rather than trying to chase dependencies the test job pulls from a candidate rhel7next and once tests pass it then publishes that repo to a location that's used by jobs and that repo is left enabled in the AMI? The test process would need to use an alternate location but same reponame and be sure to clear its caches before sealing the image.

[stevekuznetsov]

You mean instead of promoting specific packages like we do today, instead promote the entire rhel7next* ecosystem all together? We have some cards to build such a system but just have not had time to think it all through yet.

[sdodson]

@stevekuznetsov ... or mirror the repo locally to the AMI and then make the repo definition point at the local filesystem. Not sure how large the repo is or if there are any limits on the ami size. That way if the tests pass with that repo we can be sure it and all other packages that may have been part of the depsolving path are preserved.

[stevekuznetsov]

Right, I don't think it matters too much where we put it -- it just is more effort than I am willing to commit to right now.

[rhcarvalho]

Discussed with @sosiouxme on IRC that we can keep the docker_image_availability disabled for the dockertested job.

Unless anybody plans to work on the discussed solution involving yum repos, this issue can be closed.

[stevekuznetsov]

@rhcarvalho the check will need to be disabled for all jobs as when we have the newer docker installed, it will have the same conflict with skopeo-containers and the check will not be able to install skopeo from non-bleeding edge repos. If you guys want this check to be running, this is a hard pre-requisite.

[stevekuznetsov]

Actually as per this build info it looks like the %requires from Docker to skopeo-containers is not versioned, so we might be able to get away with version skew between the two packages. @lsm5 @runcom or @rhcarvalho I feel like we need to understand the dependency between these two packages better to know if we are going to be OK tolerating this skew or, as above, we are going to want to bring in skopeo from the newest compose as well