WASM builds fail on Ubuntu

[browsermt]

WASM builds fail on Ubuntu

[abhi-agg]

I am not sure what your Makefile does but if building on Ubuntu using the instructions given in README doesn't work then we can take a look 👍

[jerinphilip]

What exactly is WASM_COMPATIBLE_SOURCES vs COMPILE_WASM now?

[jerinphilip]

I did a make VERBOSE=1 to extract the command:

Context (pre)

jphilip@var:~/code/cached-builds$ BRANCH=main make wasm VERBOSE=1
mkdir -p /mnt/Storage/jphilip/bergamot /mnt/Storage/jphilip/bergamot/build /mnt/Storage/jphilip/bergamot/build/native /mnt/Storage/jphilip/bergamot/build/wasm /mnt/Storage/jphilip/bergamot/build/models.gz.d
git -C /mnt/Storage/jphilip/bergamot/bergamot-translator pull || git clone https://github.com/browsermt/bergamot-translator /mnt/Storage/jphilip/bergamot/bergamot-translator
Already up to date.
git -C /mnt/Storage/jphilip/bergamot/bergamot-translator checkout main
M       3rd_party/marian-dev
Switched to branch 'main'
Your branch is up to date with 'origin/main'.
git -C /mnt/Storage/jphilip/bergamot/bergamot-translator submodule update --init --recursive
Submodule path '3rd_party/marian-dev': checked out '370fdb5a212cfcd2d1c5fca9fffc041d2787a432'
/mnt/Storage/jphilip/bergamot/emsdk/emsdk activate latest &> /tmp/error.log || cat /tmp/error.log

source /mnt/Storage/jphilip/bergamot/emsdk/emsdk_env.sh && cd /mnt/Storage/jphilip/bergamot/build/wasm && \
        emcmake /home/jphilip/.local/bin/cmake -L \
                -DCMAKE_BUILD_TYPE=Release \
                -DCOMPILE_WASM=on \
                /mnt/Storage/jphilip/bergamot/bergamot-translator

Context (post)

Adding directories to PATH:
PATH += /mnt/Storage/jphilip/bergamot/emsdk
PATH += /mnt/Storage/jphilip/bergamot/emsdk/upstream/emscripten
PATH += /mnt/Storage/jphilip/bergamot/emsdk/node/14.15.5_64bit/bin

Is this not per instruction?

[abhi-agg]

As I said, running these build commands should work on Ubuntu,

emcmake cmake -DCOMPILE_WASM=on ../
emmake make -j

If they don't then we can look into it 👍

[jerinphilip]

Do you read what I write? Also I don't have an emmake in Ubuntu.

[jerinphilip]

source ${EMSDK}/emsdk_env.sh && \\
    emcmake cmake -L \
        -DCMAKE_BUILD_TYPE=Release \
        -DCOMPILE_WASM=on \
        .. && \
        &&  emmake make -f -j40

Here's what you see if you take effort to read through Makefile, or even the generated command. It's still a failure. Also this script worked before, it fails now.

Here's a full error log.

[jerinphilip]

Was probably an artifact of an old-ssplit compile, which left the folders with the o and didn't recompile after. I'm not sure if this triggers a CMake change as a fix wherein ssplit-cpp has to be rebuilt if COMPILE_WASM switches are on. But since this is a corner case, I'm closing the issue.

Removing all folders and rebuilding fixes the issue.

[abhi-agg]

It compiles fine for me on Linux.

Platform information:
4.15.0-132-generic #136-Ubuntu SMP Tue Jan 12 14:58:42 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Do you read what I write? Also I don't have an emmake in Ubuntu.

emmake is coming from emscripten installation. If you don't have emmake, you should never be able to compile bergamot-translator to WASM on any platform.

[jerinphilip]

emmake is irrelevant to the process (as CMake seems to generate stuff with whatever is required), it's the same as make, it works. My hypothesis is that I was experimenting with pcre2 from sources before and compiled libraries from source for Linux. Since the part (USE_INTERNAL_PCR2) didn't change, CMake/Make silently refused to recompile the (pcre2) targets and it remained for Linux, not WASM.

emscripten-core/emscripten#9988

Fresh clone solved this. This is infact a bug of sorts, but too much a corner case to not bother.