support token based authentication for Google Artifact Registry

[xelalexv]

support token based authentication for Google Artifact Registry

[xelalexv]

Need to investigate first whether Skopeo can currently work with Google artifact registry. Have you tried the Docker relay?

[mkhodave]

As skopeo is preferred choice for copying images between repositories so only tried with skopeo & I can successfully copy dockerhub images to Google artifact registry with skopeo as below command

skopeo copy docker://docker.io/library/alpine:3.12 docker://europe-west1-docker.pkg.dev/<project-id>/upstream/alpine:3.12

[xelalexv]

I tried this out with auth setting for target created in this way:

1. Create base64 encoded version of service account's key json: cat <key json> | base64 -w 0 > base64.json
2. Create auth: echo '{"username": "_json_key_base64", "password": "<content of base64.json>"}' | base64 -w 0
3. Copy output of 2) into Dregsy's config.

Sync went through without problems. The used service account has Artifact Registry Writer permissions.

[xelalexv]

With a minor change to dregsy, repos at Google Artifact Registry can now be used in the exact same way as GCR: dregsy currently determines whether we're dealing with a GCR repo based on the server URI. If this ends with .gcr.io, an auth token is retrieved from the Google API as needed. I've added -docker.pkg.dev to that, and syncing to Google Artifact Registry now works even without providing the auth setting in the dregsy task. I need to double check on that suffix though. Not sure whether that uniquely identifies Google Artifact Registry endpoints.

[mkhodave]

@xelalexv : Thanks for your troubleshooting , when we can expect release for this change ?

[mkhodave]

@xelalexv : I exact repeating your steps given for json key based authentication, following error I am observing at dregsy but can copy images using skopeo inside dregsy container (all commands run inside dregsy image running container)

# cat base64.json | skopeo login -u _json_key_base64 --password-stdin https://europe-west1-docker.pkg.dev
Login Succeeded!
# skopeo copy docker://docker.io/library/alpine:3.12 docker://europe-west1-docker.pkg.dev/<project-id>/dockerhub/alpine:3.12

Getting image source signatures
Copying blob 339de151aab4 done  
Copying config 13621d1b12 done  
Writing manifest to image destination
Storing signatures

config.yaml

--- 
relay: skopeo
skopeo: 
  binary: skopeo
  certs-dir: /etc/skopeo/certs.d
tasks: 
  - 
    mappings: 
      - 
        from: library/alpine
        to: <project-id>/dockerhub
        tags: ['3.10']
    name: dockerhub
    source: 
      auth: <dockerhub-base64-username-password-combination> 
      registry: registry.hub.docker.com
    target: 
      registry: europe-west1-docker.pkg.dev
      auth: <service-account-json-base64-username-password-combination> 
    verbose: true

dregsy -config config.yaml

INFO[0000] dregsy 0.3.6                                 
INFO[0000] skopeo version 1.2.1 commit: 1b6af3b15975f3142462b6affa85af708feb657f 
INFO[0000] relay ready                                   relay=skopeo
INFO[0000] syncing task                                  source=registry.hub.docker.com target=europe-west1-docker.pkg.dev task=dockerhub
INFO[0000] mapping                                       from=/library/alpine to=/<project-id>/dockerhub
INFO[0000] syncing tag                                   tag=3.10
DEBU[0007] Getting image source signatures              
DEBU[0011] time="2021-04-20T04:09:11Z" level=fatal msg="Error trying to reuse blob sha256:396c31837116ac290458afcb928f68b6cc1c7bdd6963fc72f52f365a2a89c1b5 at destination: Requesting bear token: invalid status code from registry 400 (Bad Request)" 
ERRO[0011] exit status 1                                
ERRO[0011] errors during sync                           
DEBU[0011] stopping tasks                               
DEBU[0011] task exited                                   task=dockerhub
DEBU[0011] exit main                                    
ERRO[0011] one or more tasks had errors, please see log for details  

am I doing something wrong ?

[xelalexv]

I've just retried my description above, and it works in my environment. Maybe you can double check by doing the following, just to make sure we really have the correct authentication info in the config:

1. Take the value of the auth setting from the dregsy config and do echo "<auth value>" | base64 -d. That should give you {"username": "_json_key_base64", "password": "<base64 encoded value>"}
2. Take the base64 encoded password value from step 1) and do echo "<base64 encoded value>" | base64 -d. The output should be identical to the contents of your key JSON file.

[mkhodave]

I have re-checked with your given steps , all looks good but still getting following error.

DEBU[0011] time="2021-04-20T04:09:11Z" level=fatal msg="Error trying to reuse blob sha256:396c31837116ac290458afcb928f68b6cc1c7bdd6963fc72f52f365a2a89c1b5 at destination: Requesting bear token: invalid status code from registry 400 (Bad Request)"

just for reference , can you please share your config.yaml sample so I can cross check there also

[xelalexv]

This is what I'm using:

relay: skopeo

tasks:
- name: test
  verbose: true
  source:
    registry: registry.hub.docker.com
  target:
    registry: europe-west1-docker.pkg.dev
    auth: <base64> 
  mappings:
  - from: library/alpine
    to: dregsy/upstream/alpine
    tags: ['3.10']

[mkhodave]

@xelalexv : I was using incorrect destination path when I changed as following then it worked perfectly

to=/<project-id>/upstream/alpine

I was not using image name in path in previous attempts,
Thanks a lot for your support

[xelalexv]

done and merged to master