Docker: go build fails due to undefined references

[gen2brain]

Docker: go build fails due to undefined references

[gen2brain]

With the latest version, it doesn't have to be so complicated.

Example for the embedded library:

FROM golang:alpine
WORKDIR /app
RUN apk add --no-cache git gcc musl-dev
COPY go.mod go.sum ./
COPY example.go test.pdf ./
RUN go mod download
RUN go build -tags musl -o /example
CMD [ "/example" ]

Example for the external library:

FROM golang:alpine
WORKDIR /app
RUN apk add --no-cache git gcc musl-dev mupdf mupdf-dev
COPY go.mod go.sum ./
COPY example.go test.pdf ./
RUN go mod download
RUN go build -tags extlib -o /example
CMD [ "/example" ]

[rntrp]

Try forcing the Golang image to Alpine 3.14: golang:1.17-alpine3.14. Image tag golang:1.17-alpine meanwhile resolves to Alpine 3.15 which has updated packages and its dependencies , i.e. mupdf 1.19.0.

I would also recommend using a multistage build version as pointed out by @m3hari because my Dockerfile is a simple proof of concept.

[rntrp]

It means that you're still running your application against a newer MuPDF version. MuPDF 1.19.0, which is part of Alpine 3.15, no longer provides /usr/lib/libmupdf-third.so.0. But the lib is included in Alpine 3.14 and MuPDF 1.18.0. Just look it up yourself:

• Contents of MuPDF under Alpine 3.14
• Contents of MuPDF under Alpine 3.15

If you're using a multistage build, I would check if your final image uses Alpine 3.14 too, i.e FROM alpine:3.14.

[gen2brain]

@mamazinho As mentioned above, you don't need all that, but if you want you can also use it like that. When you are using an external library you are on your own, there is no magic recipe, not every distro package mupdf the same way, some install only static library (with mupdf-third also), and some only use shared mupdf without third-party, they use system libs for e.g. libjpeg, freetype etc. Basic C knowledge is required whenever you deal with C libraries.

go-fitz covers all these cases with extlib, static and pkgconfig tags, so whether you need shared external lib or static, or you want to use pkg-config for your system installed mupdf, the options are there. As you see there is also an option to specify all CFLAGS/LDFLAGS manually and use whatever you want. And of course, there is also an option to use the bundled library where you don't have to do any of these things, and since recently it can also work with musl C lib.

To start, check what you are installing, whether is it shared or static and check what files you have installed and adapt to that.

[gen2brain]

You are using extlib so you must take care of all dependencies yourself. In one case you are trying with shared lib, another with static lib, and the question is how mupdf is compiled in images you use.
The only flags passed with extlib are -lmupdf -lm -lmupdf-third so if you miss anything add it to CGO_LDFLAGS.

[rntrp]

Thank you for the quick reply. If I understood you correctly, I need to try to add all dependencies needed by mupdf to the image and then call export CGO_LDFLAGS="-lfreetype ..." go build -tags extlib. I'll try it out as soon as I am back to this project.

[rntrp]

After some trial and error I was finally able to find a working configuration. @gen2brain, thank you once again for your advice. Manual dependency management and CGO_LDFLAGS actually did it.

Interestingly, CGO_LDFLAGS seems to completely override the -tags extlib flags, so that I had to explicitly set -lmupdf -lm -lmupdf-third.

Working Dockerfile for posterity:

FROM golang:1.16-alpine
RUN apk add --no-cache build-base \
    mupdf mupdf-dev \
    freetype freetype-dev \
    harfbuzz harfbuzz-dev \
    jbig2dec jbig2dec-dev \
    jpeg jpeg-dev \
    openjpeg openjpeg-dev \
    zlib zlib-dev
WORKDIR /app
COPY go.mod ./
COPY go.sum ./
COPY src/*.go ./
RUN export CGO_LDFLAGS="-lmupdf -lm -lmupdf-third -lfreetype -ljbig2dec -lharfbuzz -ljpeg -lopenjp2 -lz" \
    && go mod download \
    && go build -o /fitz-rest
EXPOSE 8080
CMD [ "/fitz-rest" ]

I was primarily interested in Alpine image, so I didn't try to build the Debian image this time. But I'm almost sure that it will work with the same packages (though they are named differently) on Bullseye/Buster.

Issue resolved. Closing.

[m3hari]

@rntrp Thank you very much! That was a huge help 🎉

[m3hari]

Here is my multistage build version for smaller image

FROM golang:1.17-alpine as builder

RUN apk add --no-cache build-base \
   mupdf-dev \
   freetype-dev \
   harfbuzz-dev \
   jbig2dec-dev \
   jpeg-dev \
   openjpeg-dev \
   zlib-dev

WORKDIR /dist
COPY go.mod ./
COPY go.sum ./
COPY src/*.go ./
RUN export CGO_LDFLAGS="-lmupdf -lm -lmupdf-third -lfreetype -ljbig2dec -lharfbuzz -ljpeg -lopenjp2 -lz" \
  && go mod download \
  && go build -o my-app


FROM  alpine:latest

RUN apk add --no-cache mupdf \
  freetype \
  harfbuzz \
  jbig2dec \
  jpeg \
  openjpeg \
  zlib

WORKDIR /app
COPY --from=builder /dist/my-app ./

ENTRYPOINT ["./my-app"]

[Ashutosh00710]

@rntrp Thanks for the dockerfile. I am trying the same dockerfile as yours but getting ⬇️ error

[+] Building 13.9s (12/12) FINISHED
 => [internal] load build definition from Dockerfile                                                                               0.0s
 => => transferring dockerfile: 32B                                                                                                0.0s
 => [internal] load .dockerignore                                                                                                  0.0s
 => => transferring context: 2B                                                                                                    0.0s
 => [internal] load metadata for docker.io/library/golang:1.17-alpine                                                              3.2s
 => [auth] library/golang:pull token for registry-1.docker.io                                                                      0.0s
 => [1/7] FROM docker.io/library/golang:1.17-alpine@sha256:1d4c7fcfd0c4c917383755ca3b6f6c1f797ec16b83cdebc672d9b08a79b64644        0.0s
 => [internal] load build context                                                                                                  0.1s
 => => transferring context: 4.73kB                                                                                                0.0s
 => CACHED [2/7] RUN apk add --no-cache build-base     mupdf mupdf-dev     freetype freetype-dev     harfbuzz harfbuzz-dev     jb  0.0s
 => CACHED [3/7] WORKDIR /app                                                                                                      0.0s
 => CACHED [4/7] COPY go.mod ./                                                                                                    0.0s
 => CACHED [5/7] COPY go.sum ./                                                                                                    0.0s
 => CACHED [6/7] COPY . .                                                                                                          0.0s
 => ERROR [7/7] RUN export CGO_LDFLAGS="-lmupdf -lm -lmupdf-third -lfreetype -ljbig2dec -lharfbuzz -ljpeg -lopenjp2 -lz"   && go  10.5s
------
 > [7/7] RUN export CGO_LDFLAGS="-lmupdf -lm -lmupdf-third -lfreetype -ljbig2dec -lharfbuzz -ljpeg -lopenjp2 -lz"   && go mod download   && go build -o pdf-transcoder:
#12 10.12 # runtime/cgo
#12 10.12 /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -lmupdf-third
#12 10.12 collect2: error: ld returned 1 exit status
------
executor failed running [/bin/sh -c export CGO_LDFLAGS="-lmupdf -lm -lmupdf-third -lfreetype -ljbig2dec -lharfbuzz -ljpeg -lopenjp2 -lz"   && go mod download   && go build -o pdf-transcoder]: exit code: 2

I am new to golang can anyone help me with that.

Dockerfile

FROM golang:1.17-alpine

RUN apk add --no-cache build-base \
    mupdf mupdf-dev \
    freetype freetype-dev \
    harfbuzz harfbuzz-dev \
    jbig2dec jbig2dec-dev \
    jpeg jpeg-dev \
    openjpeg openjpeg-dev \
    zlib zlib-dev
WORKDIR /app
COPY go.mod ./
COPY go.sum ./
COPY . .
RUN export CGO_LDFLAGS="-lmupdf -lm -lmupdf-third -lfreetype -ljbig2dec -lharfbuzz -ljpeg -lopenjp2 -lz" \
  && go mod download \
  && go build -o pdf-transcoder

CMD [ "/pdf-transcoder" ]

[Ashutosh00710]

Worked for me. Thanks @rntrp 🚀

[Ashutosh00710]

@rntrp After running the image got this error

Error loading shared library libmupdf-third.so.0: No such file or directory

[Ashutosh00710]

It worked for me 🚀. Big thanks for the quick reply @rntrp and rescuing me from this situation.

[mamazinho]

The problem stiil happen on go 1.19

[gen2brain]

@mamazinho Which problem exactly, there were different issues here, and almost all were user issues.

[mamazinho]

Downloading all external libs and passing the flags to the build, occurs an error on ld command

[mamazinho]

with go1.19 image

[mamazinho]

It was a version problem. We used on our project an old version (1.18.0).
Updating the versionthe problem not occured anymore. Thank you

[Deme94]

Hey, I couldn't make it work with alpine, I tried most of these solutions but they didn't work for me, I probably did something wrong, but I want to share the solution that worked for me. The same way it works in my ubuntu, this is just a debian image so I find it easier (it is not alpine sadly).

FROM golang:1.21.1-bullseye

ENV CGO_ENABLED=1
RUN apt update && apt install -y build-essential

WORKDIR /app

COPY go.mod go.sum ./
RUN go mod download

COPY . .

RUN go build -o ./out/app ./main.go

CMD ["./out/app"]

[gen2brain]

Use musl tag for alpine. If you search the issues you will find many examples.

[Deme94]

With the latest version, it doesn't have to be so complicated.

Example for the embedded library:

FROM golang:alpine
WORKDIR /app
RUN apk add --no-cache git gcc musl-dev
COPY go.mod go.sum ./
COPY example.go test.pdf ./
RUN go mod download
RUN go build -tags musl -o /example
CMD [ "/example" ]

Example for the external library:

FROM golang:alpine
WORKDIR /app
RUN apk add --no-cache git gcc musl-dev mupdf mupdf-dev
COPY go.mod go.sum ./
COPY example.go test.pdf ./
RUN go mod download
RUN go build -tags extlib -o /example
CMD [ "/example" ]

You are right, this worked for me, I just changed it to work with project, sorry for the confusion.

Thank you