urbit / vere Goto Github PK

An implementation of the Urbit runtime

License: MIT License

C 96.69% Shell 0.11% Starlark 3.00% Nix 0.21%

c interpreter nock runtime vm

vere's Introduction

Urbit

Urbit is a personal server stack built from scratch. It has an identity layer (Azimuth), virtual machine (Vere), and operating system (Arvo).

A running Urbit "ship" is designed to operate with other ships peer-to-peer. Urbit is a general-purpose, peer-to-peer computer and network.

This repository contains the Arvo Kernel

For the Runtime, see Vere. For more on the identity layer, see Azimuth. To manage your Urbit identity, use Bridge.

Install

To install and run Urbit, please follow the instructions at urbit.org/getting-started. You'll be on the live network in a few minutes.

Contributing

Contributions of any form are more than welcome! Please take a look at our contributing guidelines for details on our git practices, coding styles, and how we manage issues.

You might also be interested in joining the urbit-dev mailing list.

Release

For details about our release process, see the maintainers guidelines

vere's People

Contributors

Stargazers

Watchers

vere's Issues

Use `bazel`'s remote caching feature

Describe the bug
https://bazel.build/remote/caching

Spin up UF-owned `macos-aarch64` and `linux-aarch64` runners for GitHub Actions

We currently use @mcevoypeter's macos-aarch64 and linux-aarch64 boxes as self-hosted runners for builds, but we should use our own. We can use MacStadium for the former and BuildJet for the latter platforms.

Use `bazel` caching to improve CI performance

Use bazel caching to improve automated build and test performance.

Build with `bazel` on `linux-arm64`

Build the urbit binary on linux-arm64. This will build off the work done in #2. The steps required:

Tweak third-party dependency bazel build rules in bazel/third_party/ to build on linux-arm64 platforms.
Tweak the bazel build rules for all packages in pkg/ to build on linux-arm64 platforms.

Build with `bazel` on `openbsd-x86_64`

Build the urbit binary on openbsd-x86_64. This will build off the work done in #2. The steps required:

Tweak third-party dependency bazel build rules in bazel/third_party/ to build on openbsd-x86_64 platforms.
Move the platform-specific third-party dependency patches in pkg/urbit/compat/openbsd/ to the appropriate dependency directory in bazel/third_party/; verify the patches are still correct in the context of the dependency versions we're using; and apply the patches to the dependency builds in WORKSPACE.bazel using select().
Migrate the platform-specific files in pkg/urbit/compat/openbsd/ to pkg/noun/platform/openbsd/x86_64/ or pkg/vere/platform/openbsd/x86_64/ (whichever is more appropriate) and incorporate them in the appropriate build rule(s).
Tweak the bazel build rules for all packages in pkg/ to build on openbsd-x86_64 platforms.

Self-host M1 macOS GitHub Actions runner

To test the macos-arm64 build in CI, we need an M1 macOS machine. Unfortunately, GitHub Actions doesn't provide them by default, so we need to set up a self-hosted option instead.

Replace `arm64` with `aarch64` as canonical identifier for 64-bit ARM architecture

We've been using arm64 to identify the 64-bit ARM architecture in the build system, but they are the same, and aarch64 appears to be more widely used.

Build with `bazel` on `darwin-x86_64`

Build the urbit binary on darwin-x86_64 (i.e. x86_64 machines running macOS). This will build off the work done in #2. The steps required:

Tweak the third-party dependency bazel build rules in bazel/third_party/ to build on darwin-x86_64 platforms.
Tweak the bazel build rules for all packages in pkg/ to build on darwin-x86_64 platforms.

Resolve URL discrepancies in binary upgrade system and CI

The binary upgrade system and CI don't currently use the exact same URLs. The binary upgrade system expects URLs of the form {ver_hos_c}/{pace}/{version_number}/vere-v{version_number}-{os}-{cpu}, but CI uploads binaries to {ver_hos_c}/{pace}/v{version_number}/vere-v{version_number}-{os}-{cpu} (note the leading "v" before the first {version_number}). What's worse, CI uses a shortened commit SHA in place of the version number for the edge train. To fix this, the binary upgrade system should add a leading "v" before the first {version_number} in its URL, and CI should upload using an actual version number to the edge train, not a commit SHA.

Automatically create GitHub releases

Use the gh CLI to create a new GitHub release as part of the existing release flow in our CI code. GitHub releases should include:

a compressed archive for each of our supported platforms,
release notes,
and a list of contributions/commits since last release.

Test release of 1.16

We should perform a test release of 1.16 to ensure the release flow works as expected. Of chief importance is ensuring that a fake ship running 1.15 can use the next subcommand to get the 1.16.

macos: `bazel build //pkg/vere:boot-fake-ship` produces `libsigsegv (stackoverflow_deinstall_handler): Cannot allocate memory`

Describe the bug
On both macos-x86_64 and macos-arm64, booting a ship works without error when executing the urbit binary from the console. However, when trying to boot a fake ship from within bazel (with bazel build //pkg/vere:boot-fake-ship), a libsigsegv error prints during the boot phase:

matt@mbp14 vere % bazel build //pkg/vere:boot-fake-ship
...
libsigsegv (stackoverflow_deinstall_handler): Cannot allocate memory
...

The ship finishes booting and seems to behave normally from the dojo, however. Also, when re-booting the ship from the resulting pier, no libsigsegv (or other) errors appear.

To Reproduce
Steps to reproduce the behaviour:

bazel build //pkg/vere:boot-fake-ship

Expected behaviour
No error message.

System (please supply the following information, if relevant):

OS: macOS Ventura 13.0
Vere 1.13

Additional context

See pkg/noun/manage.c for additional context.

Notify maintainers
@mcevoypeter @joemfb

CI action logs are too noisy

Describe the bug
For example: https://github.com/urbit/vere/actions/runs/3847814390/jobs/6554760836

75%+ of the logs are relatively unhelpful tar lines printed during musl installation.

Run HTTP client IO driver in a separate process

The HTTP client IO driver should be rewritten (in Rust) to run in a separate process to improve security by sandboxing one of Urbit's interfaces to the outside world and to enable future scalability. The steps required:

Re-implement _cttp_ef_http_client() (the HTTP client driver function defined in pkg/vere/io/cttp.c that releases effects) in Rust in the IO drivers repo, rendering the existing driver implementation redundant.
Replace the now-redundant HTTP client driver C implementation with an implementation that asynchronously writes a request to the new HTTP client process's stdin and reads any subsequent response asynchronously from the HTTP client proccess's stdout.

Both Scaling Urbit's Runtime and Scalable IO provide additional relevant context for this work.

Build with `bazel` on `mingw-x86_64`

Build the urbit binary on mingw-x86_64. This will build off the work done in #2. The steps required:

Tweak third-party dependency bazel build rules in bazel/third_party/ to build on mingw-x86_64 platforms.
Move the platform-specific third-party dependency patches in pkg/urbit/compat/mingw/ to the appropriate dependency directory in bazel/third_party/; verify the patches are still correct in the context of the dependency versions we're using; and apply the patches to the dependency builds in WORKSPACE.bazel using select().
Migrate the platform-specific files in pkg/urbit/compat/mingw/ to pkg/noun/platform/mingw/x86_64 or pkg/vere/platform/mingw/x86_64 (whichever is more appropriate) and incorporate them in the appropriate build rule(s).
Tweak the bazel build rules for all packages in pkg/ to build on mingw-x86_64 platforms.

Build with `bazel` on `darwin-arm64`

Build the urbit binary on darwin-arm64 (i.e. Apple Silicon machines running macOS). This will build off the work done in #2. The steps required:

Tweak third-party dependency bazel build rules in bazel/third_party/ to build on darwin-arm64 platforms.
Move the platform-specific third-party dependency patches in pkg/urbit/compat/m1brew/ to the appropriate dependency directory in bazel/third_party/; verify the patches are still correct in the context of the dependency versions we're using; and apply the patches to the dependency buidls in WORKSPACE.bazel using select().
Incorporate the compilation flags in pkg/urbit/compat/m1brew/compat.mk into the appropriate build rule(s) in pkg/.
Tweak the bazel build rules for all packages in pkg/ to build on darwin-arm64 platforms.

Build with `bazel`

Build the runtime using bazel for each of the following supported platforms:

Document build system's dependency on `automake` and `libtool` on Linux

When building on Linux, the automake and libtool suite of tools need to be installed, just as they do on macOS. INSTALL.md should call attention to this.

Migrate from Urbit monorepo

Pre-migration checklist

Post-migration checklist

Remove all runtime-related files and references in urbit/urbit.
Clearly communicate to runtime developers that they need to file an issue against this repo for anything that appears broken in the new repo.
Collect list of open runtime-related PRs in urbit/urbit and re-open them in this repo.

Fakeship tests fail on macOS builds

https://github.com/urbit/vere/actions/runs/3715632920/jobs/6301027661

Same errors on macos-x86_64 and macos-arm64.

Uncomment definition of U3_OS_ARCH on macos-aarch64

The definition of U3_OS_ARCH on macos-aarch64 is commented out in pkg/c3/portable.h. Now that we have a native macos-aarch63 build, this should be uncommented.

Point binary upgrade system to bootstrap.urbit.org/vere-test

Change the URL prefix the runtime binary upgrade system uses from https://bootstrap.urbit.org/vere to https://bootstrap.urbit.org/vere-test to test the release pipeline system.

Correct faulty default pill path

The runtime attempts to download the default pill from https://bootstrap.urbit.org/props/{version}/brass.pill, but the default pill actually lives at https://bootstrap.urbit.org/urbit-v{version}.pill.

Reduce CI log noise/verbosity on `build static binary` and `run tests` jobs

See this recent job for reference.

Document debugging flow

We should include in INSTALL.md on how to produce a debug binary via the use of --copt="-g" along with other useful debugging related information.

Review and update continuous deploy flow

Upon completion of the new bazel-based CI flow (see #8), we need to review and likely update our continuous deploy flow in .github/workflows/release.yml for releasing new versions of the urbit binary.

This requires that #8 is complete.

Migrate `git` history from Urbit monorepo

Migrate all runtime-related commits from next/vere in the Urbit monorepo. The last commit shared by this repo and the Urbit monorepo is df9a0e7c1d317cf556dc9752e348009d90d3ce99, which means we need all commits in next/vere committed after df9a0e7c1d317cf556dc9752e348009d90d3ce99.

Build and release Docker images

Use bazelbuild/rules_docker to build a linux/amd64 Docker image automatically when new releases are tagged. Upload (release) images with a GitHub Actions workflow.

Reference
https://github.com/urbit/urbit/blob/master/.github/workflows/release-docker.yml

Select proper system include paths on macOS

The fix for #23 introduced in #24 does not work on a macOS install that has the Xcode application installed because Bazel detects a dependency on /Applications/Xcode.app/Contents/Developer instead of on /Library/Developer/CommandLineTools. The proper sys_includes paths for macOS really depend on the output of xcode-select --print-path, but Bazel makes it difficult to consume version information from arbitrary binaries like xcode-select. A possible work-around is to force Bazel to ignore /Applications/Xcode.app using the --sandbox_block_path flag and therefore force it to use /Library/Developer/CommandLineTools instead.

Ensure GitHub workflow caches aren't flushed

For some reason (probably a simple size limit rule), our Bazel and musl caches for our CI jobs are getting flushed, making build times very slow when they don't have to be.

Overwrite binaries deployed to `soon` and `edge` trains

The binaries deployed to https://bootstrap.urbit.org/vere/{edge|soon}/v{version_number}/vere-v{version_number}-{platform} should be overwritten as we update the tip of develop (the branch tracked by the edge train) and release (the branch tracked by the soon train) in preparing a release. Binaries deployed to the live train should never be overwritten.

Correctly cast `hun_y` when trimming newlines in `u3_king_next()`

hun_y is a c3_y *, but strrchr() expects a c3_c *. We should cast hun_y to c3_c *.

Remove support for `openbsd` and `windows` platforms

`argon2`, `gmp`, and `softfloat` have "no symbols" (linker warnings on `macos-x86_64`)

Describe the bug

"has no symbols" linker warnings for these libraries occur when building with bazel build //... on macos-x86_64:

INFO: From Linking external/argon2/libargon2.a:
/Library/Developer/CommandLineTools/usr/bin/ranlib: file: bazel-out/darwin-fastbuild/bin/external/argon2/libargon2.a(thread.o) has no symbols
...
INFO: From Linking external/softfloat/softfloat.a:
/Library/Developer/CommandLineTools/usr/bin/ranlib: file: bazel-out/darwin-fastbuild/bin/external/softfloat/libsoftfloat_x86_64.a(s_countLeadingZeros16.o) has no symbols
/Library/Developer/CommandLineTools/usr/bin/ranlib: file: bazel-out/darwin-fastbuild/bin/external/softfloat/libsoftfloat_x86_64.a(s_countLeadingZeros32.o) has no symbols
/Library/Developer/CommandLineTools/usr/bin/ranlib: file: bazel-out/darwin-fastbuild/bin/external/softfloat/libsoftfloat_x86_64.a(s_countLeadingZeros64.o) has no symbols
/Library/Developer/CommandLineTools/usr/bin/ranlib: file: bazel-out/darwin-fastbuild/bin/external/softfloat/libsoftfloat_x86_64.a(s_mul64ByShifted32To128.o) has no symbols
/Library/Developer/CommandLineTools/usr/bin/ranlib: file: bazel-out/darwin-fastbuild/bin/external/softfloat/libsoftfloat_x86_64.a(s_mul64To128.o) has no symbols
/Library/Developer/CommandLineTools/usr/bin/ranlib: file: bazel-out/darwin-fastbuild/bin/external/softfloat/libsoftfloat_x86_64.a(s_mul128By32.o) has no symbols
/Library/Developer/CommandLineTools/usr/bin/ranlib: file: bazel-out/darwin-fastbuild/bin/external/softfloat/libsoftfloat_x86_64.a(s_mul128To256M.o) has no symbols

Expected behaviour

The libraries don't generate symbols in these files on linux-x86_64 either, so there's no error. We'd just like to suppress the warnings.

Additional context

I will add more context next week.

Set up `bazel`-based continuous integration flow

Create a new GitHub action .github/workflows/bazel-build.yml that builds all targets via bazel build ... and runs all test targets via bazel test ... for all supported platforms. For the list of platforms we support, see #1. The existing nix-based continuous integration flow in .github/workflows/build.yml may serve as a useful reference point.

This feature requires that #1 is complete.

Correct upload of Docker image

This CI job turned up issues with //:upload_docker. The tag attribute of container_push takes a string, not a label. Instead of the string_flag image_tag, we should instead use a genrule that generates a file with the appropriate container tag from pkg/vere/VERSION and pkg/vere/PACE.

New dill & terminal changes

Containing urbit/urbit#4463 and urbit/urbit#5663.

Upload the latest binary version number to GCP

The latest binary version number needs to be uploaded to {url_base}/{pace}/v{version_number}/last.

Switch from tag-based release flow to push-based release flow

To simplify the release flow, we should switch from a tag-based flow to a push-based release flow. In the new scheme, we'll have a release branch, which has all work that is tested, reviewed, and ready to go out in the next release. All feature work will target the release branch. Any time a commit is pushed to a feature branch--any branch that is not master or release--all unit and integration tests will run via the feature flow. Once a feature is ready to be staged for release, the feature's PR will merge into release, at which point the pre-release flow will run, triggering binary uploads for the soon train to bootstrap.urbit.org. Once a release is ready to go out, the version number in pkg/vere/VERSION will be bumped and release will be rebased on top of master, triggering the release flow to run, which will re-run all unit and integration tests and--assuming the tests pass--trigger binary uploads for the live train to bootstrap.urbit.org.

Fix musl libc toolchain cache access

We install the musl libc toolchains (for linux-x86_64 and linux-arm64) in /usr/local, which causes an issue with the GitHub cache action because writing to /usr/local requires sudo. To get around this, we should install the musl libc toolchains in ~.

Build for `macos-x86_64` without sandboxing

Due to a dependency resolution error¹ when building for macos-x86_64 on macos-12 GitHub Actions hosted runner machines, we use --spawn_strategy=sandboxed and --sandbox_block_path=/usr/local/include to build successfully in our CI workflow. Unfortunately, using Bazel's sandboxing feature effectively disables caching, resulting in macos-x86_64 build/test times of usually 15+ minutes. Comparatively, our builds for linux-arm64 and linux-x86_64 (which both utilize caching) usually complete in under 3 minutes.

We need to determine why the dependency resolution error occurs and implement a solution which doesn't use sandboxing. Once done, our macos-x86_64 builds will automatically begin to benefit from caching in the same way the other two target's builds do.

https://github.com/urbit/vere/actions/runs/3586593418/jobs/6035968777 ↩

Temporarily disable fake-ship tests

The fake ship tests take ~20 minutes to run, which interferes with the ability to rapidly iterate on CI-related changes that don't affect the functionality of the runtime.

Use `git` tags exclusively for setting the release version

Describe the bug
Currently we use a combination of git tags and the pkg/vere/VERSION file to denote release version numbers. To simplify the workflow, developers should only have to tag a new release in order to set the URBIT_VERSION macro and upload binaries and Docker images with the correct version labels.

Additional context
See:

Point binary upgrade system back to https://bootstrap.urbit.org/vere

#58 (via #60 ) pointed the binary upgrade system to a test endpoint--https://bootstrap.urbit.org/vere-test. Once we complete testing the new release pipeline, we should point the binary upgrade system back to https://bootstrap.urbit.org/vere.

`cc_test` targets fail to build for `linux-arm64`

Description

All cc_test targets fail to build when targeting linux-arm64 because of a failure to detect a suitable toolchain. Toolchain resolution for tests apparently differs from library and binary targets, presumably because tests are built for the execution platform rather than the target platform by default.

Reproduction Steps

To reproduce, run the following on an x86_64 machine with Linux and version 12.2.0 of aarch64-linux-gnu-gcc installed:

% bazel build --aarch64_linux_gnu_gcc_version="12.2.0" --host_platform=//:aarch64_linux_gnu_gcc-linux-x86_64 --platforms=//:linux-arm64 --toolchain_resolution_debug='.*' ...
INFO: Build option --toolchain_resolution_debug has changed, discarding analysis cache.
INFO: ToolchainResolution: Target platform //:linux-arm64: Selected execution platform //:aarch64_linux_gnu_gcc-linux-x86_64,
INFO: ToolchainResolution: Target platform //:linux-arm64: Selected execution platform //:aarch64_linux_gnu_gcc-linux-x86_64,
INFO: ToolchainResolution:   Type @bazel_tools//tools/cpp:toolchain_type: target platform //:linux-arm64: execution //:aarch64_linux_gnu_gcc-linux-x86_64: Selected toolchain //bazel/toolchain:aarch64_linux_gnu_gcc-linux-x86_64
INFO: ToolchainResolution:     Type @bazel_tools//tools/cpp:toolchain_type: target platform //:linux-arm64: Rejected toolchain //bazel/toolchain:clang-linux-x86_64; mismatching values: x86_64
INFO: ToolchainResolution:     Type @bazel_tools//tools/cpp:toolchain_type: target platform //:linux-arm64: Rejected toolchain //bazel/toolchain:gcc-linux-x86_64; mismatching values: x86_64
INFO: ToolchainResolution:   Type @bazel_tools//tools/cpp:toolchain_type: target platform //:linux-arm64: execution //:aarch64_linux_gnu_gcc-linux-x86_64: Selected toolchain //bazel/toolchain:aarch64_linux_gnu_gcc-linux-x86_64
INFO: ToolchainResolution:     Type @bazel_tools//tools/cpp:toolchain_type: target platform //:linux-arm64: Rejected toolchain //bazel/toolchain:clang-linux-x86_64; mismatching values: x86_64
INFO: ToolchainResolution:     Type @bazel_tools//tools/cpp:toolchain_type: target platform //:linux-arm64: Rejected toolchain //bazel/toolchain:gcc-linux-x86_64; mismatching values: x86_64
INFO: ToolchainResolution: Target platform //:linux-arm64: Selected execution platform //:aarch64_linux_gnu_gcc-linux-x86_64, type @bazel_tools//tools/cpp:toolchain_type -> toolchain //bazel/toolchain:aarch64_linux_gnu_gcc-linux-x86_64
INFO: ToolchainResolution: Target platform //:linux-arm64: Selected execution platform //:aarch64_linux_gnu_gcc-linux-x86_64, type @bazel_tools//tools/cpp:toolchain_type -> toolchain //bazel/toolchain:aarch64_linux_gnu_gcc-linux-x86_64
INFO: ToolchainResolution: Target platform //:aarch64_linux_gnu_gcc-linux-x86_64: Selected execution platform //:aarch64_linux_gnu_gcc-linux-x86_64,
INFO: ToolchainResolution: Target platform //:linux-arm64: Selected execution platform //:aarch64_linux_gnu_gcc-linux-x86_64,
INFO: ToolchainResolution:   Type @rules_foreign_cc//toolchains:m4_toolchain: target platform //:linux-arm64: execution //:aarch64_linux_gnu_gcc-linux-x86_64: Selected toolchain @rules_foreign_cc//toolchains:preinstalled_m4
INFO: ToolchainResolution:   Type @rules_foreign_cc//toolchains:automake_toolchain: target platform //:linux-arm64: execution //:aarch64_linux_gnu_gcc-linux-x86_64: Selected toolchain @rules_foreign_cc//toolchains:preinstalled_automake
INFO: ToolchainResolution: Target platform //:aarch64_linux_gnu_gcc-linux-x86_64: Selected execution platform //:aarch64_linux_gnu_gcc-linux-x86_64,
INFO: ToolchainResolution:   Type @rules_foreign_cc//toolchains:pkgconfig_toolchain: target platform //:linux-arm64: execution //:aarch64_linux_gnu_gcc-linux-x86_64: Selected toolchain @rules_foreign_cc//toolchains:preinstalled_pkgconfig
INFO: ToolchainResolution:   Type @rules_foreign_cc//toolchains:make_toolchain: target platform //:linux-arm64: execution //:aarch64_linux_gnu_gcc-linux-x86_64: Selected toolchain @rules_foreign_cc//toolchains:preinstalled_make
INFO: ToolchainResolution:   Type @rules_foreign_cc//toolchains:autoconf_toolchain: target platform //:linux-arm64: execution //:aarch64_linux_gnu_gcc-linux-x86_64: Selected toolchain @rules_foreign_cc//toolchains:preinstalled_autoconf
INFO: ToolchainResolution:   Type @rules_foreign_cc//foreign_cc/private/framework:shell_toolchain: target platform //:linux-arm64: execution //:aarch64_linux_gnu_gcc-linux-x86_64: Selected toolchain @rules_foreign_cc_framework_toolchain_linux//:commands
INFO: ToolchainResolution: Target platform //:linux-arm64: Selected execution platform //:aarch64_linux_gnu_gcc-linux-x86_64, type @rules_foreign_cc//toolchains:autoconf_toolchain -> toolchain @rules_foreign_cc//toolchains:preinstalled_autoconf, type @rules_foreign_cc//toolchains:m4_toolchain -> toolchain @rules_foreign_cc//toolchains:preinstalled_m4, type @rules_foreign_cc//foreign_cc/private/framework:shell_toolchain -> toolchain @rules_foreign_cc_framework_toolchain_linux//:commands, type @bazel_tools//tools/cpp:toolchain_type -> toolchain //bazel/toolchain:aarch64_linux_gnu_gcc-linux-x86_64, type @rules_foreign_cc//toolchains:pkgconfig_toolchain -> toolchain @rules_foreign_cc//toolchains:preinstalled_pkgconfig, type @rules_foreign_cc//toolchains:automake_toolchain -> toolchain @rules_foreign_cc//toolchains:preinstalled_automake, type @rules_foreign_cc//toolchains:make_toolchain -> toolchain @rules_foreign_cc//toolchains:preinstalled_make
INFO: ToolchainResolution:     Type @bazel_tools//tools/cpp:toolchain_type: target platform //:aarch64_linux_gnu_gcc-linux-x86_64: Rejected toolchain //bazel/toolchain:aarch64_linux_gnu_gcc-linux-x86_64; mismatching values: arm64
INFO: ToolchainResolution:     Type @bazel_tools//tools/cpp:toolchain_type: execution platform //:aarch64_linux_gnu_gcc-linux-x86_64: Rejected toolchain //bazel/toolchain:clang-linux-x86_64; mismatching values: clang
INFO: ToolchainResolution:     Type @bazel_tools//tools/cpp:toolchain_type: execution platform //:aarch64_linux_gnu_gcc-linux-x86_64: Rejected toolchain //bazel/toolchain:gcc-linux-x86_64; mismatching values: gcc
INFO: ToolchainResolution:   Type @bazel_tools//tools/cpp:toolchain_type: target platform //:aarch64_linux_gnu_gcc-linux-x86_64: No toolchains found.
ERROR: /home/tlon/.cache/bazel/_bazel_tlon/d85bfd16708db53fe097086a278bf2d0/external/remote_coverage_tools/BUILD:9:12: While resolving toolchains for target @remote_coverage_tools//:Main: No matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type. Maybe --incompatible_use_cc_configure_from_rules_cc has been flipped and there is no default C++ toolchain added in the WORKSPACE file? See https://github.com/bazelbuild/bazel/issues/10134 for details and migration instructions.
ERROR: Analysis of target '//pkg/noun:nock_tests' failed; build aborted:
INFO: Elapsed time: 0.271s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded, 1225 targets configu\
red)

Make release pipeline resilient to newline in `pkg/vere/VERSION`

If pkg/vere/VERSION ends in a newline, the binary upgrade system fails with the following:

% zod/.run next
~
urbit 1.15
boot: home is /home/tlon/code/gh/urbit/new-vere/zod
loom: mapped 2048MB
lite: arvo formula 11a9e7fe
lite: core 38d4ad4d
lite: final state 38d4ad4d
loom: mapped 2048MB
boot: protected loom
live: loaded: MB/54.853.632
boot: installed 351 jets
vere: next (%edge): 1.16

curl: failed https://bootstrap.urbit.org/vere-test/edge/v1.16
/vere-v1.16
-linux-x86_64: Error
unable to save https://bootstrap.urbit.org/vere-test/edge/v1.16
/vere-v1.16
-linux-x86_64 to /home/tlon/code/gh/urbit/new-vere/zod/.bin/edge/vere-v1.16
-linux-x86_64: -1
vere: upgrade failed
vere: prep for upgrade failed

Build with musl instead of glibc on Linux

The urbit binary should link against musl, which supports static linking, instead of glibc. When linked against glibc, an urbit binary may fail to run on a different machine than the one it was compiled on, even if both machines share an OS and architecture, because of mismatches in the glibc version. Statically linking against musl addresses this problem. Unfortunately, musl availability is confined to Linux because it's built atop the Linux system call API.¹

musl-cross-make: a tool for compiling musl-targeting compilers

See https://musl.libc.org/about.html. ↩

Build with `bazel` on `linux-x86_64`

Build the urbit binary on linux-x86_64. The steps required:

Build all third-party dependencies from source using bazel.
Build the urcrypt package using bazel.
Split the urbit package into three separate packages--c3, noun, and vere--with clearly defined, directed dependency relationships between the three.

Replace Xcode-specific `sys_includes` path in `clang-macos-arm64` toolchain

The current include paths for the clang-macos-arm64 toolchain require the Xcode is installed:

sys_includes = [
    "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include",
    "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/{compiler_version}/include",

     ],

These paths should instead be changed to paths that only require the developer command line tools to be installed:

sys_includes = [
    "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include",
    "/Library/Developer/CommandLineTools/usr/lib/clang/{compiler_version}/include",
]

Run file system IO driver in a separate process

The file system IO driver should be rewritten (in Rust) to run in a separate process to improve security by sandboxing one of Urbit's interfaces to the outside world and to enable future scalability. The steps required:

Re-implement the following functions (in pkg/vere/io/unix.c) that release file system effects in Rust in the IO drivers, rendering the existing driver implementation redundant:
a. u3_unix_ef_dirk(): commit a new mount point.
b. u3_unix_ef_ergo(): update the file system
c. u3_unix_ef_ogre(): delete an existing mount point.
d. u3_unix_ef_hill(): scan for new mount points.
Replace the now-redundant file system driver C implementation with an implementation that asynchronously write a request to the new file system process's stdin and reads any subsequent response asynchronously from the file system proccess's stdout. Note that this implementation is nearly identical to the same implementation required for the HTTP client driver in #11; the two implementations should be unified into a single implementation.