Doesn't play nice with `docker-compose`

[buildkite-plugins]

Doesn't play nice with `docker-compose`

[lox]

That is very odd @jjmilburn! The ecr plugin basically just calls docker login under the hood and should work exactly as you describe 🤔 My first thought is that for some reason the account id's or regions aren't matching up between the two, but some troubleshooting is needed.

Mind dropping us a link to the failing build to [email protected] and we will have a look?

[jjmilburn]

Will do, thank you!

@lox I dropped an email to [email protected] and can continue to engage there or here - either way.

[orien]

I've seen this behavior when the AWS role used by the Buildkite agent doesn't have access to the AWS ECR: the login succeeds, but the cannot access the repository.

Worth a look to see if it's the cause of your problem.

[TSMMark]

EDIT: PEBCAK. I was missing the leading ci: in front of cache-from: <ACCOUNT>.dkr.ecr.us-west-2.amazonaws.com/d2-ci:latest. It should have been cache-from: ci:<ACCOUNT>.dkr.ecr.us-west-2.amazonaws.com/d2-ci:latest

---

Original post below

I'm having this same/similar issue. For the record, the IAM role has been granted full ECR access, as a sanity check, and still does not pull from ECR, although it pushes to ECR totally fine...

Here's the relevant part of my pipeline.yml:

steps:
  - label: ":docker: Build an image"
    plugins:
      - ecr#v2.1.1:
          login: true
          region: us-west-2
          no-include-email: true
      - docker-compose#v3.7.0:
          build: ci
          image-repository: <ACCOUNT>.dkr.ecr.us-west-2.amazonaws.com/d2-ci
          cache-from: <ACCOUNT>.dkr.ecr.us-west-2.amazonaws.com/d2-ci:latest
          push:
            - ci:<ACCOUNT>.dkr.ecr.us-west-2.amazonaws.com/d2-ci
            - ci:<ACCOUNT>.dkr.ecr.us-west-2.amazonaws.com/d2-ci:latest

AFAICT it should be working but it fails to pull from the repo, even tho the images are there and tagged latest. Here's the log of successful ECR login:

Logs successful ECR login:
Running plugin ecr environment hook
[2020-10-16T14:35:18Z] $ var/lib/buildkite-agent/plugins/github-com-buildkite-plugins-ecr-buildkite-plugin-v2-1-1/hooks/environment
[2020-10-16T14:35:19Z] WARNING! Your password will be stored unencrypted in /var/lib/buildkite-agent/.docker/config.json.
[2020-10-16T14:35:19Z] Configure a credential helper to remove this warning. See
[2020-10-16T14:35:19Z] https://docs.docker.com/engine/reference/commandline/login/#credentials-store
[2020-10-16T14:35:19Z]
[2020-10-16T14:35:19Z] Login Succeeded

But then the ECR pull fails, saying I may need docker login:

:docker: Pulling cache image for <ACCOUNT>.dkr.ecr.us-west-2.amazonaws.com/d2-ci
[2020-10-16T14:35:24Z] $ docker pull latest
[2020-10-16T14:35:25Z] Using default tag: latest
[2020-10-16T14:35:26Z] Error response from daemon: pull access denied for latest, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
[2020-10-16T14:35:26Z] Exited with 1
[2020-10-16T14:35:26Z] !!! :docker: Pull failed. latest will not be used as a cache for <ACCOUNT>.dkr.ecr.us-west-2.amazonaws.com/d2-ci

The repo + image definitely exist in ECR, and are in the us-west-2 region: <ACCOUNT>.dkr.ecr.us-west-2.amazonaws.com/d2-ci:latest

As @lox said, I should not need a to define a separate docker login plugin, since ecr should be using it under the hood anyway. Also I'm able to push to ECR fine, confirming that ecr login is working, at least in that case.

Any advice would be a huge help, thank you in advance.