Giter Site home page Giter Site logo

goxx's Introduction

GitHub release Build Status Test Status Docker Stars Docker Pulls

Become a sponsor Donate Paypal


About

This repo contains a Dockerfile for building an image which can be used as a base for building your Go project using CGO. All the necessary Go tool-chains, C/C++ cross-compilers and platform headers/libraries can be installed with the specially crafted wrappers. This project is heavily inspired by xx project.

Projects using goxx

Docker image

Registry Image
Docker Hub crazymax/goxx
GitHub Container Registry ghcr.io/crazy-max/goxx
$ docker run --rm mplatform/mquery crazymax/goxx:latest
Image: crazymax/goxx:latest
 * Manifest List: Yes
 * Supported platforms:
   - linux/amd64
   - linux/arm64

Supported platforms

Platform CC CXX
darwin/amd64¹ o64-clang o64-clang++
darwin/arm64¹ o64-clang o64-clang++
linux/386 i686-linux-gnu-gcc i686-linux-gnu-g++
linux/amd64 x86_64-linux-gnu-gcc x86_64-linux-gnu-g++
linux/arm64 aarch64-linux-gnu-gcc aarch64-linux-gnu-g++
linux/arm/v5 arm-linux-gnueabi-gcc arm-linux-gnueabi-g++
linux/arm/v6 arm-linux-gnueabi-gcc arm-linux-gnueabi-g++
linux/arm/v7 arm-linux-gnueabihf-gcc arm-linux-gnueabihf-g++
linux/mips² mips-linux-gnu-gcc mips-linux-gnu-g++
linux/mipsle² mipsel-linux-gnu-gcc mipsel-linux-gnu-g++
linux/mips64² mips64-linux-gnuabi64-gcc mips64-linux-gnuabi64-g++
linux/mips64le² mips64el-linux-gnuabi64-gcc mips64el-linux-gnuabi64-g++
linux/ppc64le powerpc64le-linux-gnu-gcc powerpc64le-linux-gnu-g++
linux/riscv64 riscv64-linux-gnu-gcc riscv64-linux-gnu-g++
linux/s390x s390x-linux-gnu-gcc s390x-linux-gnu-g++
windows/386 i686-w64-mingw32-gcc i686-w64-mingw32-g++
windows/amd64 x86_64-w64-mingw32-gcc x86_64-w64-mingw32-g++

¹ darwin* platform requires the MacOSX cross toolchain if using CGO.

² compilers for mips* archs are not available with linux/arm64 image.

Usage

In order to use this image effectively, we will use the docker buildx command. Buildx is a Docker component that enables many powerful build features. All builds executed via buildx run with Moby BuildKit builder engine.

# syntax=docker/dockerfile:1

FROM --platform=$BUILDPLATFORM crazymax/goxx:1.17 AS base
ENV GO111MODULE=auto
ENV CGO_ENABLED=1
WORKDIR /go/src/hello

FROM base AS build
ARG TARGETPLATFORM
RUN --mount=type=cache,sharing=private,target=/var/cache/apt \
  --mount=type=cache,sharing=private,target=/var/lib/apt/lists \
  goxx-apt-get install -y binutils gcc g++ pkg-config
RUN --mount=type=bind,source=. \
  --mount=type=cache,target=/root/.cache \
  --mount=type=cache,target=/go/pkg/mod \
  goxx-go build -o /out/hello ./hello.go

FROM scratch AS artifact
COPY --from=build /out /

FROM scratch
COPY --from=build /out/hello /hello
ENTRYPOINT [ "/hello" ]
  • FROM --platform=$BUILDPLATFORM ... command will pull an image that will always match the native platform of your machine (e.g., linux/amd64). BUILDPLATFORM is part of the ARGs in the global scope.
  • ARG TARGETPLATFORM is also an ARG in the global scope that will be set to the platform of the target that will default to your current platform or can be defined via the --platform flag of buildx so goxx-* wrappers will be able to automatically build against the right platform.

More details about multi-platform builds in this blog post.

Let's run a simple build against the artifact target in our Dockerfile:

# build and output content of the artifact stage that contains the binaries in ./dist
docker buildx build \
  --platform "linux/amd64,linux/arm64,linux/arm/v7,darwin/amd64" \
  --output "./dist" \
  --target "artifact" .

$ tree ./dist
./dist
├── darwin_amd64
│ ├── hello
├── linux_amd64
│ ├── hello
├── linux_arm64
│ ├── hello
├── linux_arm_v7
│ ├── hello

You can also create a multi-platform image in addition to the artifacts:

docker buildx build \
  --platform "linux/amd64,linux/arm64,linux/arm/v7" \
  --tag "hello:latest" \
  --push .

More examples can be found in the examples folder.

Build

Build goxx yourself using Docker buildx bake:

git clone https://github.com/crazy-max/goxx.git goxx
cd goxx

# create docker container builder
docker buildx create --name goxx --use

# build goxx image and output to docker with goxx:local tag (default)
docker buildx bake image-local

# examples
(cd ./examples/c ; docker buildx bake artifact-all)
(cd ./examples/cpp ; docker buildx bake artifact-all)
(cd ./examples/echo ; docker buildx bake artifact-all)
(cd ./examples/gorm ; docker buildx bake artifact-all)
(cd ./examples/jq ; docker buildx bake artifact-all)

Notes

MacOSX cross toolchain

You can use the MacOSX cross toolchain provided by crazymax/osxcross image to build against the darwin platform with CGO.

Using the COPY command:

FROM --platform=$BUILDPLATFORM crazymax/osxcross:11.3 AS osxcross
FROM base AS build
COPY --from=osxcross /osxcross /osxcross
ARG TARGETPLATFORM
RUN --mount=type=bind,source=. \
  --mount=type=cache,target=/root/.cache \
  --mount=type=cache,target=/go/pkg/mod \
  goxx-go build -o /out/hello ./hello.go

Or a RUN mount:

FROM --platform=$BUILDPLATFORM crazymax/osxcross:11.3 AS osxcross
FROM base AS build
ARG TARGETPLATFORM
RUN --mount=type=bind,source=. \
  --mount=from=osxcross,target=/osxcross,src=/osxcross,rw \
  --mount=type=cache,target=/root/.cache \
  --mount=type=cache,target=/go/pkg/mod \
  goxx-go build -o /out/hello ./hello.go

Wrappers

Wrappers are a significant part of this repo to dynamically handle the build process with a go wrapper named goxx-go which will automatically sets values for GOOS, GOARCH, GOARM, GOMIPS, etc. but also AR, CC, CXX if building with CGO based on defined ARGs in the global scope like TARGETPLATFORM.

It also manages debian packages for cross-compiling for non-native architectures with goxx-apt-get and goxx-macports to install MacPorts packages.

CGO

By default, CGO is enabled in Go when compiling for native architecture and disabled when cross-compiling. It's therefore recommended to always set CGO_ENABLED=0 or CGO_ENABLED=1 when cross-compiling depending on whether you need to use CGO or not.

Install cross-compilers for all platforms

In some case you may want to install cross-compilers for all supported platforms into a fat image:

# syntax=docker/dockerfile:1-labs

ARG PLATFORMS="linux/386 linux/amd64 linux/arm64 linux/arm/v5 linux/arm/v6 linux/arm/v7 linux/mips linux/mipsle linux/mips64 linux/mips64le linux/ppc64le linux/riscv64 linux/s390x windows/386 windows/amd64"

FROM --platform=$BUILDPLATFORM crazymax/goxx:1.17 AS base
ARG PLATFORMS
RUN <<EOT
export GOXX_SKIP_APT_PORTS=1
goxx-apt-get update
for p in $PLATFORMS; do
  TARGETPLATFORM=$p goxx-apt-get install -y binutils gcc g++ pkg-config
done
EOT

Note: This is not recommended for production use.

Contributing

Want to contribute? Awesome! The most basic way to show your support is to star the project, or to raise issues. You can also support this project by becoming a sponsor on GitHub or by making a Paypal donation to ensure this journey continues indefinitely!

Thanks again for your support, it is much appreciated! 🙏

License

MIT. See LICENSE for more details.

goxx's People

Contributors

0xjacky avatar crazy-max avatar dependabot[bot] avatar fabio1988 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Forkers

0xjacky

goxx's Issues

goxx-apt-get install doesn't resolve dpkg-dev when building linux/amd64 -> linux/arm64

Support guidelines

I've found a bug and checked that ...

  • ... the documentation does not mention anything about my problem
  • ... there are no open or closed issues that are related to my problem

Description

My application requires pkg-config due to the use of CGO, however when building, I get this error:

#12 23.76 Please install dpkg-dev to use pkg-config when cross-building
#12 23.76 aarch64-linux-gnu-pkg-config: exit status 1

My dockerfile calls it out as a requirement:

FROM --platform=$BUILDPLATFORM docker-hub.repo.splunkdev.net/crazymax/goxx AS build
ENV CGO_ENABLED=1
WORKDIR /go/src/app
ARG TARGETPLATFORM
ARG BINARY_NAME
ARG VERSION
RUN goxx-apt-get update && \
    goxx-apt-get install -y binutils gcc dpkg-dev pkg-config libpcsclite-dev # dpkg-dev specified here
RUN --mount=type=bind,source=. \
  --mount=type=secret,id=_netrc,dst=/root/.netrc \
  goxx-go env && goxx-go build -v -o /out/$BINARY_NAME -ldflags="-X main.Version=$VERSION" .

FROM scratch AS artifact
COPY --from=build /out/$BINARY_NAME /

FROM scratch
COPY --from=build /out/$BINARY_NAME /
ENTRYPOINT [ "/$BINARY_NAME" ]

But when goxx-apt-get install runs, it excludes dpkg-dev:

#10 7.507 Reading package lists...
#10 22.30 + exec apt-get install -y binutils-aarch64-linux-gnu gcc-aarch64-linux-gnu pkg-config-aarch64-linux-gnu libpcsclite-dev:arm64

It's worth noting that this same dockerfile works on a linux/arm64 node.

Expected behaviour

Building on a linux/amd64 host for a linux/arm64 binary succeeds.

Actual behaviour

Building on a linux/amd64 host for a linux/arm64 binary fails.

Steps to reproduce

See description

Docker info

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.4
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.17.2
    Path:     /usr/libexec/docker/cli-plugins/docker-compose
Server:
 Containers: 3
  Running: 1
  Paused: 0
  Stopped: 2
 Images: 33
 Server Version: 23.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 31aa4358a36870b21a992d3ad2bef29e1d693bec
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
 Kernel Version: 5.4.0-1101-aws
 Operating System: Ubuntu 18.04.5 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 15.35GiB
 Name: runner-sfxftlit-production-1684440269-ee1b8f70
 ID: WD4W:EZTB:G4Z7:HNNT:T3U7:YLGM:E4AM:4HBI:CI5R:5G6T:U7T5:TJ5E
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
  provider=amazonec2
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
WARNING: No swap limit support

Logs

See description

Additional info

This is running in Gitlab CI using the docker+machine executor

Support platform=js/wasm

Description

yet new to using "goxx" (started yesterday), I am not yet familiar with the internals. So I am not aware how much effort it would take to implement "js/wasm" as an additional target platform.

However it would be great if "webassembly" (js/wasm) would also be supported.

An implementation or hints to relevant places in the code would be appreciated.

[doc][enhancement] Add windows Arm support in doc

Hi,

Great work ! and thanks for sharing :)

I saw that windows arm is supported (and I tested windows/arm64), but the readme do not offer this precision.

Suggestion :
update readme to include windows arm support

Many thanks 👍

error: failed to solve: no match for platform in manifest when using multi-stage builds

I've got a node.js frontend that I serve with a go daemon that handles a REST API. I discovered this project because I need to have this daemon work on OSX and goxx does a fantastic job of compiling my CGO app for all the supported platforms. The issue is trying to incorporate the node.js build into goxx.

Here are the relevant bits from my Dockerfile:

ARG GOXX_BASE
ARG GOBUILDTYPE
FROM node:latest as node-build

WORKDIR /base
COPY pixelweb/companion/package.json .
COPY pixelweb/companion/yarn.lock .
RUN --mount=type=bind,source=. \
  cd pixelweb/companion && yarn install
RUN --mount=type=bind,source=. \
  cd pixelweb/companion && yarn build

When trying to build, I now get the following error:

 => CANCELED [linux/amd64 internal] load metadata for docker.io/crazymax/goxx:latest                                                                                           0.1s
 => ERROR [windows/amd64 internal] load metadata for docker.io/library/node:latest                                                                                             0.1s
 => CANCELED [linux/amd64 internal] load metadata for docker.io/crazymax/osxcross:11.3                                                                                         0.1s
 => CANCELED [windows/386 internal] load metadata for docker.io/library/node:latest                                                                                            0.1s
------
 > [windows/amd64 internal] load metadata for docker.io/library/node:latest:
------
Dockerfile:4
--------------------
   2 |     ARG GOXX_BASE
   3 |     ARG GOBUILDTYPE
   4 | >>> FROM node:latest as node-build
   5 |
   6 |     WORKDIR /base
--------------------
error: failed to solve: no match for platform in manifest sha256:ad3314b09371eb7f085c689ab2a3b00e41362cbc20e98bfaa27a9eaf78abd2a8: not found

I'm assuming this is because goxx is trying to select a node container matching the target platform? This issue is preventing the project from moving fully to goxx. I currently have to run yarn install && yarn build in another container and then build the CGO app using the already built node.js app.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.