support ARM and ARM64 architecture?

[ldez]

support ARM and ARM64 architecture?

[ldez]

@andig I created https://github.com/ldez/seihon, then if you want to test it 😉

[ldez]

Hello, the 2 arch are available as binaries: https://github.com/ldez/traefik-certs-dumper/releases/tag/v2.2.0

Multi-arch images on the Docker Hub is a little bit complex to create.

To build the image you have to get the code inside in the same directory.

[kamilmirza]

yes I am using binaries for now

To build the image you have to get the code inside in the same directory

oh yes, apologies, will try again with clone

[andig]

As discussed on slack with @ldez I've whipped up a strawman for building multiarch images and bundling inside a manifest. It basically builds and pushes images for all platforms and in a second step bundles and publishes a manifest.

I've tested this on OSX but- since the runtime image is only COPYied into and does not use RUN- this build process should work on any platform without need to qemu cross-build support. Thanks to golang's cross-compiling capability the builder image does not need to be platform specific.

Currently the script uses jq as external dependency which could be replaced by docker run realguess/jq jq. Docker is available as travis service, too.

build-docker.sh

#!/bin/bash
set -e

# base docker image tag
TAG="andig/traefik-certs-dumper"

# only linux for now
OS=linux

# target platforms
declare -a PLATFORMS=( "amd64" "arm64" "arm.v6" )

# images from Dockerfile
RUNTIME_IMAGE=$(grep "FROM alpine" < Dockerfile | sed "s/FROM //")

# platform-specific docker file
DOCKER_TEMP=/tmp/Dockerfile.temp

# cache the docker manifest
function manifest () {
    local IMAGE=$1
    local MANIFEST_FILE=/tmp/manifest.$IMAGE.json

    if [ ! -f "$MANIFEST_FILE" ]; then
        docker pull "$IMAGE"
        docker manifest inspect "$IMAGE" > "$MANIFEST_FILE"
    fi
}

# get platform image hash from docker manifest
function hash () {
    local IMAGE=$1
    local MANIFEST_FILE=/tmp/manifest.$IMAGE.json

    local ARCHITECTURE="$2"
    local VARIANT="$3"
    local HASH

    if [ -z "$VARIANT" ]; then
        HASH=$(jq -r ".manifests[] | select(.platform.architecture == \"$ARCHITECTURE\") | .digest" < "$MANIFEST_FILE")
    else
        HASH=$(jq -r ".manifests[] | select(.platform.architecture == \"$ARCHITECTURE\" and .platform.variant == \"$VARIANT\") | .digest" < "$MANIFEST_FILE")
    fi

    echo "$HASH"
}

# get manifests
manifest "$RUNTIME_IMAGE"

# main
for platform in "${PLATFORMS[@]}"; do 
    # split architecture.version
    IFS='.' read -ra p <<< "$platform"

    # add xargs to trim whitespace
    RUNTIME_HASH=$(hash "$RUNTIME_IMAGE" "${p[0]}" "${p[1]}")

    # target architecture
    GOARCH=${p[0]}

    # create temp Dockerfile for target architecture and build
    sed "s/RUN make/RUN GOARCH=$GOARCH make/" Dockerfile | sed "s/$RUNTIME_IMAGE/$RUNTIME_IMAGE@$RUNTIME_HASH/" > "$DOCKER_TEMP"
    docker build -t "$TAG:latest-$platform" -f $DOCKER_TEMP .
    rm "$DOCKER_TEMP"
done

# push images
for platform in "${PLATFORMS[@]}"; do 
    docker push "$TAG:latest-$platform"
done

# create manifest
TAG_LIST=$(printf "$TAG:latest-%s " "${PLATFORMS[@]}")
# shellcheck disable=SC2086
docker manifest create --amend "$TAG:latest" $TAG_LIST

for platform in "${PLATFORMS[@]}"; do 
    # split architecture.version
    IFS='.' read -ra p <<< "$platform"

    ARCHITECTURE=${p[0]}
    VARIANT=${p[1]}
    
    # docker and go architectures don't match
    if [ "arm" == "$ARCHITECTURE" ] && [ ! -z "$VARIANT" ]; then
        VARIANT=$ARCHITECTURE$VARIANT
    fi

    if [ -z "$VARIANT" ]; then
        docker manifest annotate "$TAG:latest" "$TAG:latest-$platform" --os "$OS" --arch "$ARCHITECTURE"
    else
        docker manifest annotate "$TAG:latest" "$TAG:latest-$platform" --os "$OS" --arch "$ARCHITECTURE" --variant "$VARIANT"
    fi
done

docker manifest push "$TAG:latest"

The result is available as andig/traefik-certs-dumper:latest for demo purposes. This is intentionally not a PR as I didn't know how to integrate it with the goreleaser.

I imagine one would run build-docker.sh inside travis after_sucess like this:

.travis.yml

env:
  global:
    - PATH=/home/travis/gopath/bin:$PATH
    # docker credentials
    - secure: "MMDlT1..."
    - secure: "aFjBMVNpdv..."
    - COMMIT=${TRAVIS_COMMIT::8}

after_success:
  - docker login -u $DOCKER_USER -p $DOCKER_PASS
  - export REPO=andig/gravo
  - docker pull $REPO:latest
  - |
    if [ "$TRAVIS_BRANCH" == "master" ]; then 
      ./build-docker.sh
    fi

docker credentials are added to travis using the travis cli or through travis ui.

[andig]

@ldez I've simplified the build script some and added a PR for reference.

Getting rid of the sed magic required adding docker ARGs which is why I've added the Dockerfile in addition to the build script. Also added arm architecture variants to the build

Hopefully you'll find it useful.

[ldez]

The version v2.3.4 now support multi-arch Docker image:

• arm v6
• arm v7
• arm v8 (arm64)
• 386
• amd64

[andig]

I love the go-based publisher. Do you think there would be additional value in publishing that as its own repo or even multi-arch docker image? Something like go-multiarch-publisher? Would be happy to test drive a separate component.