Can the source of mappings use a pattern ?

[xelalexv]

Can the source of mappings use a pattern ?

[xelalexv]

I'm currently finishing up a basic test set. Having this in place is a prerequisite for adding new features. Once that is done, pattern based mapping and filtering is the first feature on my list. I have a few ideas for that, but really can't say when I'll get this done. I hope I get some time over the holidays.

[xelalexv]

There is an alpha implementation available now in branch filter. To try it out, first have a look at the design document, which gives an overview of the approach and a few examples. There's also a Docker image xelalex/dregsy:filter over at DockerHub. But remember, it's still alpha and I'm sure it will need a bit more work. Nevertheless, basic functionality is there.

[xelalexv]

That's currently not possible, but would be really useful. I'll look into it.

[yannh]

@fredgate it's interesting, though how would you suggest the to & tags fields to work in that context?

[evandam]

Hi folks,

Any updates on this by any chance? I was looking at https://github.com/bdudelsack/docker-registry-mirror which allows passing a regex to match tags in the source. It can be a little clunky but it can allow for some really powerful stuff.

[fredbcode]

I trying to test xelalex/dregsy:filter but unfortunately my CA bundle is not available when regex is unable (only with regex)

X509 certificate signed by unknown authority

I tried to add my AC with update-certificates without more success

from regex:test/* -> ko
from test/docker -> ok

relay:docker

[fredbcode]

Same with skip-tls-verify: true and skopeo
For information I'm using proxy, but source and target are in NO_PROXY env value and works well without regex

EDIT: same with proxy removed from system/docker

[xelalexv]

Could you post your config?

[fredbcode]

relay: skopeo

skopeo:
  # path to the skopeo binary; defaults to 'skopeo', in which case it needs to
  # be in PATH
  binary: skopeo
  # directory under which to look for client certs & keys, as well as CA certs
  # (see note below)
  certs-dir: /etc/docker/certs.d

docker:
  # Docker host to use as the relay
  dockerhost: unix:///var/run/docker.sock
  # Docker API version to use, defaults to 1.24
  api-version: 1.24

tasks:
  - name: task1
    interval: 60
    source:
      registry: a.fr
      auth: xxxxxxxxxxxxxx
      skip-tls-verify: true
    target:
      registry: b.fr
      auth: xxxxxxxxxxxxxx
      skip-tls-verify: true
    mappings:
        - from: regex:images/.*
           to: images/internet/
           tag: latest
 # works
 #     - from: images/docker
 #      to: images/internet/docker
 #       tag: latest
 #     - from: images/traefik
 #       to: images/internet/traefik
 #       tag: latest

[xelalexv]

Can you also post a log, with the task set to verbose, and env var LOG_LEVEL=debug? Also, does the registry you're using support the v2/_catalog API? That's the default lister, and is being used here since you're not specifying a different lister in source.

[xelalexv]

I think I may have found the problem. The catalog lister was not minding the skip-tls-verify property of the source. This is fixed. I pushed an update to the filter image.

Edit: Above is of course only a solution if you can live with skip-tls-verify. To use proper TLS verification with custom CAs not covered by the bundle present in the dregsy container, update-certificates is not the right path. Have a look at section Repository Validation & Client Authentication with TLS in the README.

[fredbcode]

Ok I will test soon, I'm out of office

I missed the syntax about lister in documentation ?
I just try to add lister in block ?

    source:
      registry: a.fr/
      lister: v2/_catalog

Yes about the CA, I just tried with update-certificates also, I added the CA in /etc/docker/certs.d but without more success, maybe there is something wrong because I just want make a quick test with skip-tls-verify

[xelalexv]

Since this is still an alpha feature not yet merged into master, there is no documentation in the top level README. Please have a look at the design document. This gives detailed explanation & examples on how to use image matching.

Re custom CAs, please make sure you're creating proper sub-folders underneath /etc/docker/certs.d/ for Docker relay, and /etc/skopeo/certs.d for Skopeo relay. The top level README has detailed info on that. There are a few subtleties to watch out for, such as sub-folder name and cert file ending.

[fredbcode]

I'm using latest clair image, v2 api is enabled

I will check and debug

[fredbcode]

Could you share your email please ? I will send to you more informations about my tests
numsys - @ - free - fr

[xelalexv]

merged as alpha-feature into master