ocurrent/ocaml-multicore-ci

OCurrent CI for multicore OCaml trees

OCaml-Multicore-CI

This is an OCurrent pipeline that provides CI for OCaml projects hosted on GitHub, using multicore variants of the OCaml compiler.

This is currently a fork of ocaml-ci. We expect to merge features into ocaml-ci or OCurrent over time, so that this repo becomes thinner. Refer to ocaml-ci and OCurrent documentation for general architecture and design decisions.

The pipeline is defined in pipeline.ml. It:

1. Gets the list of installations of its GitHub app.
2. For each installation, gets the list of repositories to check.
3. For each repository, gets the branches and PRs to check.
4. For each target, it fetches the head commit, generates a Dockerfile and builds it.

The generated Dockerfile first adds all the *.opam files found in the project, then uses opam to install all the dependencies, then adds the rest of the source files. This means that rebuilds are often very fast, because Docker will reuse the previously cached build step as long as the opam files don't change.

To add the CI to your own project:

1. Go to https://github.com/apps/multicore-ci and install the app for your GitHub user.
2. Configure just the repositories you want to test (start with one!).
3. Ask us to add you to the alpha-testers list by submitting a PR against this repository adding yourself to --github-account-allowlist in stack.yml.
4. Report bugs :-)

Installation

Get the code with:

git clone --recursive https://github.com/ocurrent/ocaml-multicore-ci.git

Note: you need to clone with --recursive because this project uses submodules (it depends on some packages that aren't released yet). If you forget, git submodule update --init will fetch them.

To test the CI on a local Git clone, you need to first install the dependencies, and then use dune exec as shown below:

opam update
opam install . --deps-only --yes --with-test
dune exec -- ocaml-multicore-ci-local /path/to/project

This will build the project as the real CI would, but it only monitors the default branch and does not push the results anywhere. It runs a web interface at http://localhost:8080. This is useful if you want to try out changes to the pipeline.

If you want to build the whole system, the easiest way is using Docker:

docker build -t ocaml-multicore-ci-service .
docker build -f Dockerfile.web -t ocaml-multicore-ci-web .

You'll have to register your own GitHub app to be able to test the services locally.

If you want it to update to changes in opam-repository automatically you'll also need to register a webhook there sending push events to the CI's /webhooks/github path.

Remote API

The service provides a Cap'n Proto endpoint and a command-line client that uses it. You will need to be given the ocaml-ci.cap file, which grants access to the API. The client can be built and run using dune exec -- ocaml-multicore-ci --ci-cap=ocaml-ci.cap ..., or installed as ocaml-multicore-ci.

To see the branches and PRs that ocaml-multicore-ci is monitoring in a repository:

$ ocaml-multicore-ci mirage/irmin
615364620f4233cb82a96144824eb6ad5d1104f0 refs/heads/1.4
e0fcf0d336544650ca5237b356cfce4a48378245 refs/heads/master
6c46d1de5e67a3f504fc55af1d644d852c946533 refs/heads/mirage-dev
28421a152e8e19b3fb5048670629e7e01d0fbea6 refs/pull/523/head
acfbee7e82fcaaa5a0dad900068dc67f22021f2e refs/pull/678/head
3fc04e9f6e7574c0f61eacb3187b412b3bababe4 refs/pull/728/head
32f6c9f303616880994998881ee75c8d1fe0df91 refs/pull/771/head
b2d4b06f94d13384ae08eb06439ce9c6066419cd refs/pull/815/head
d8161e6cbf06c3005a080d4df209f7de67d6fa5c refs/pull/851/head
5e36237d7ce6279878578cf48d8b63937c499e5a refs/pull/858/head
04a368ecd52ea436bfcd252ed94772f55b5159d5 refs/pull/866/head
2e838b491a4c0b21750f7a2e6dee88eee1c7d94e refs/pull/867/head

You can pass either the reference (e.g. refs/heads/master) or the commit hash to choose one of them.

$ ocaml-multicore-ci mirage/irmin refs/heads/master
alpine-3.10-ocaml-4.08

To view the log (following it if incomplete):

$ ocaml-multicore-ci mirage/irmin refs/heads/master alpine-3.10-ocaml-4.08 log
[...]
- Test Successful in 17.643s. 99 tests run.
-> compiled  irmin-unix.dev
-> installed irmin-unix.dev
Done.
# Run eval $(opam env) to update the current shell environment
Removing intermediate container 4c85cdc76ddc
 ---> c8e34c3b5eee
Successfully built c8e34c3b5eee
2019-09-25 14:55.57: Job succeeded

Instead of log, you can also use cancel, rebuild or status.

For convenience, you can omit the leading refs/ when specifying a reference, and for PRs you can omit the trailing /head. For commits, you must give at least the first 6 characters. e.g.

$ ocaml-multicore-ci mirage/irmin pull/867 alpine-3.10-ocaml-4.08 cancel

ocaml-multicore-ci's Issues

Different use case between Sandmark and Multicore-CI

Bench/Sandmark has different use case between the current Multicore-CI:

• In the case of Multcore-CI

  • Take a github repo and build all the packages within it.
  • All repo are build with the recent version of dune
  • Analyzing a package take account all other package within the same repo with the package that we want to analyze.

• In the case of Sandmark dependencies:

  • First thing sandmark has some opam dependencies packages in it own repos.
  • The github repo of some packages has already another packages and Sandmark not need to build them for their CI.
  • The packages need to be built with specific dune version (not the last version).
  • The analyze part could be an issue because some repo could have many packages.

---

Job failure on `Uncaught exception: (Failure "file_name can't be split")`

cf. https://multicore.ci.ocamllabs.io:8100/job/2021-12-13/104333-ci-ocluster-build-c62725

cc @moyodiallo

Backport changes from Ewan's fork to ocurrent

These two changes should be PR'd against the main repo:

• https://github.com/ewanmellor/ocurrent/commit/6f78bc37d8af49b11407f8996cb054f00da8a128
• https://github.com/ewanmellor/ocurrent/commit/ed64f49dcdb849b8e55d13d185a8ffa9a10c60c4

Link in web UI is broken

Reported by @ctk21:

I’ve tried clicking on The OCaml-Multicore-CI GitHup App link and get a 404 error

It goes to https://github.com/apps/ocaml-multicore-ci but should probably be https://github.com/apps/multicore-ci (although the first name does sound better, really).

Tezos job timeout

Currently (on commit f5d080c) a job's timeout is set to 7 hours. The Tezos job on 4.12.0+domains occasionally surpasses this time limit resulting in a failed job. (cf. https://multicore.ci.ocamllabs.io:8100/job/2021-12-12/233831-ci-ocluster-build-effa23)

While 7 hours is a reasonably lengthy time limit, we should also take into consideration that the python integration tests that are part of Tezos' test suite in itself take close to 4 hours. Before attempting other means, it might be useful to increase the time limit to see if that results in completing the test-suite run.

cc @moyodiallo

Two Tezos jobs use same network port at same time

Hi
This issue is about when Tezos test runs at same time on 4.12.0 and 4.12.0+domains, when the network port is used by one run and the other can't because of the same port.

┌──────────────────────────────────────────────────────────────────────────────┐
│ [FAIL]        p2p-connection-pool          0   simple.                       │
└──────────────────────────────────────────────────────────────────────────────┘
idtrCxjv3sZz:55011: PID: 90231
idtkRJrL9kyd:55012: PID: 90232
idrMpPQSJvjj:55013: PID: 90233
idqdTzCbGwVN:55014: PID: 90234
idsfoLeD3FnX:55015: PID: 90235
idqhVnG4DM3Q:55016: PID: 90236
idrLqLUQrdPJ:55017: PID: 90237
ids7Em5CA9eM:55018: PID: 90238
idtHthjZZWXj:55019: PID: 90239
idqn6hcKfRGe:55020: PID: 90240
idtkRJrL9kyd:55012: Detached process ended with error.Error:
idtkRJrL9kyd:55012:                                     An error occured while initializing P2P server on this address: 127.0.0.1:55012.
idtkRJrL9kyd:55012:                                       Reason: Address already in use.
idtkRJrL9kyd:55012:                                       Another tezos node is probably running on this address.
idtkRJrL9kyd:55012:                                       Please choose another P2P port using --net-addr.
idtkRJrL9kyd:55012: 
idtkRJrL9kyd:55012: 
idqdTzCbGwVN:55014: Detached process ended with error.Error:
idqdTzCbGwVN:55014:                                     An error occured while initializing P2P server on this address: 127.0.0.1:55014.
idqdTzCbGwVN:55014:                                       Reason: Address already in use.
idqdTzCbGwVN:55014:                                       Another tezos node is probably running on this address.
idqdTzCbGwVN:55014:                                       Please choose another P2P port using --net-addr.
idqdTzCbGwVN:55014: 
idqdTzCbGwVN:55014: 
Nov 24 15:23:02.696 - process: Early error! Canceling remaining process.
Nov 24 15:23:02.696 - process: The processes 0(idtrCxjv3sZz:55011:) 2(idrMpPQSJvjj:55013:)
Nov 24 15:23:02.696 - process:               3(idqdTzCbGwVN:55014:) 4(idsfoLeD3FnX:55015:)
Nov 24 15:23:02.696 - process:               5(idqhVnG4DM3Q:55016:) 6(idrLqLUQrdPJ:55017:)
Nov 24 15:23:02.696 - process:               7(ids7Em5CA9eM:55018:) 8(idtHthjZZWXj:55019:)
Nov 24 15:23:02.696 - process:               9(idqn6hcKfRGe:55020:) 
Nov 24 15:23:02.696 - process:      have been canceled.
Nov 24 15:23:02.696 - process: The process 1(idtkRJrL9kyd:55012:) 
Nov 24 15:23:02.696 - process:     failed with error:
Nov 24 15:23:02.696 - process:     Error:
Nov 24 15:23:02.696 - process:       An error occured while initializing P2P server on this address: 127.0.0.1:55012.
Nov 24 15:23:02.696 - process:         Reason: Address already in use.
Nov 24 15:23:02.696 - process:         Another tezos node is probably running on this address.
Nov 24 15:23:02.696 - process:         Please choose another P2P port using --net-addr.
Nov 24 15:23:02.696 - process: 
Nov 24 15:23:02.696 - process: 
Nov 24 15:23:02.696 - process: 
[failure] Error:
  The processes 0(idtrCxjv3sZz:55011:) 2(idrMpPQSJvjj:55013:)
                3(idqdTzCbGwVN:55014:) 4(idsfoLeD3FnX:55015:)
                5(idqhVnG4DM3Q:55016:) 6(idrLqLUQrdPJ:55017:)
                7(ids7Em5CA9eM:55018:) 8(idtHthjZZWXj:55019:)
                9(idqn6hcKfRGe:55020:) 
       have been canceled.
  The process 1(idtkRJrL9kyd:55012:) 
      failed with error:
      Error:
        An error occured while initializing P2P server on this address: 127.0.0.1:55012.
          Reason: Address already in use.
          Another tezos node is probably running on this address.
          Please choose another P2P port using --net-addr.

I discussed with @talex5, he noticed to me it's caused by the parameter network host of a run (oBuilder). With this generated script we can see the parameter:

(run (cache (opam-archives (target /home/opam/.opam/download-cache)))
                   (network host)
                   (shell "opam exec -- bash -c 'source $HOME/.poetry/env; source $HOME/.cargo/env; make build-deps && eval $(opam env); PATH=\"$HOME/.local/bin:$PATH\"; make -C tests_python install-dependencies && make && make test'"))

The configuration is in this file

ocaml-multicore-ci/service/conf.ml

Line 147 in 3c06080

| "tezos" -> `Script ["sudo apt-get install -y rsync git m4 build-essential patch unzip wget pkg-config libgmp-dev libev-dev libhidapi-dev libffi-dev opam jq zlib1g-dev bc autoconf software-properties-common"; "gpg --keyserver keyserver.ubuntu.com --recv-keys BA6932366A755776"; "gpg --export BA6932366A755776 | sudo apt-key add -"; "sudo add-apt-repository 'deb http://ppa.launchpad.net/deadsnakes/ppa/ubuntu bionic main'"; "sudo apt-get update"; "sudo apt-get install -y python3.9 python3.9-dev python3.9-distutils python3-pip virtualenv python3.9-venv"; "wget https://sh.rustup.rs/rustup-init.sh"; "chmod +x rustup-init.sh"; "./rustup-init.sh --profile minimal --default-toolchain 1.52.1 -y"; "opam install dune"; "curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python3.9 -"; "python3.9 -m pip install --upgrade setuptools"; "bash -c 'source $HOME/.poetry/env; source $HOME/.cargo/env; make build-deps && eval $(opam env); PATH=\"$HOME/.local/bin:$PATH\"; make -C tests_python install-dependencies && make && make test'"]

There's no alternative to avoid the parameter network host for solving this issue when configuring(config.ml) because Script variant is generated with.

Build Sandmark dependency packages with Multicore OCaml CI

The Sandmark benchmark suite has a number of dependency packages that are needed by the benchmark programs.

• The pinned version of the dependency packages are listed in the dev.opam file. It will be useful to know if these packages continue to build using the Multicore OCaml CI.
  https://github.com/ocaml-bench/sandmark/blob/main/dependencies/template/dev.opam
• There are few other dependencies/packages that are locally maintained, which we are also upstreaming. If the same can be added to Multicore OCaml CI, it will be useful.
• If there are newer package versions of the above in opam.ocaml.org, we would also like to know if they build fine, so that we can update our dependencies.

README needs updating

It still refers to ocaml-ci.

Some "opam-repository" commits used in the analysis are not during the build

During the analysis it's possible to have more than one "opam-repository" commits (like opam-repository and opam-alpha-repository. After the analysis only the default one (opam-repository) is used during the build, that cause an issue when the analyzer find a suitable package in other repositories against the default one.

Status ID conflicts with ocaml-ci

If ocaml-ci and ocaml-multicore-ci are enabled on the same repository, they fight over the ocaml-ci status value.

Job links in the UI have the wrong URL.

Currently clicking on a build in the multicore UI has a URL with an extra slash.
eg
https://multicore.ci.ocamllabs.io:8100//job/2022-05-30/213156-ci-ocluster-build-6ae122
should be
https://multicore.ci.ocamllabs.io:8100/job/2022-05-30/213156-ci-ocluster-build-6ae122