f0rmiga / gcc-toolchain Goto Github PK
View Code? Open in Web Editor NEWA fully-hermetic Bazel GCC toolchain for Linux.
License: Apache License 2.0
A fully-hermetic Bazel GCC toolchain for Linux.
License: Apache License 2.0
What happened: linking a Cgo library into a Go binary produces a warning about incorrect use of DT_TEXTREL in a PIE target.
What I expected: no warnings, because consistent link strategies are used for intermediate and final results.
A Go package that uses cgo, compiled as a library in debug mode, will normally produce a .gopclntab that requires relocations. When using the standard Go rules with a system C toolchain on various flavors of Ubuntu, cgo=True
libraries are built such that .gopclntab
and other relocated read-only sections are prefixed with .data.relro
. It appears that some interaction between the Aspect GCC toolchain and the Go rules is leading to a situation where intermediate .o
files for Go builds are using internal linkage, while the final link step is using GNU ld
as an external linker.
That conflict causes warnings and has in the past been a sign of binaries with executable data sections that need to be specially handled by the system loader. Notably, various musl-based systems are not able to correctly load such binaries.
One possible cause here is that the PIE-by-default nature of the GCC toolchain is not correctly detected by the Go rules.
See golang/go issue 10914, 17847, and 59866 for related discussions.
Development (host) and target OS/architectures:
Host is Ubuntu 22.04.3 LTS, uname -srv
Linux 6.2.0-1018-gcp #20~22.04.1-Ubuntu SMP Mon Oct 23 12:29:43 UTC 2023
Target is Linux x86-64, with similar kernel revisions
Output of bazel --version
: bazel 6.4.0
Version of the Aspect rules, or other relevant rules from your
WORKSPACE
or MODULE.bazel
file: @io_bazel_rules_go
= v0.43.0, Go toolchain = 1.20.2, Aspect GCC toolchain = 0.4.2
Complete repo available at https://github.com/rbosetti/aspect-repro
Language(s) and/or frameworks involved: Go, C
`bazel build //:demobin` from the workspace in https://github.com/rbosetti/aspect-repro
Output from the repro step.
INFO: Analyzed target //:demobin (55 packages loaded, 16855 targets configured).
INFO: Found 1 target...
INFO: From GoLink demobin_/demobin:
/home/rlb/.cache/bazel/_bazel_rlb/5d655e0d5670b9e1307c236b8abb2e79/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /tmp/go-link-2297262172/go.o: warning: relocation in read-only section `.gopclntab'
/home/rlb/.cache/bazel/_bazel_rlb/5d655e0d5670b9e1307c236b8abb2e79/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: warning: creating DT_TEXTREL in a PIE
Target //:demobin up-to-date:
bazel-bin/demobin_/demobin
INFO: Elapsed time: 191.332s, Critical Path: 166.08s
INFO: 13 processes: 7 internal, 6 linux-sandbox.
INFO: Build completed successfully, 13 total actions
I'm working on integrating support for gcc 12.2.0. I added the relevant entries to _TOOLCHAINS
in toolchain/defs.bzl
and built an updated sysroot, also updating _SYSROOTS
in the same file to point to my local archive. Things work mostly as expected, however, there is one issue when compiling a C source file that includes stdatomic.h
. Since gcc 12.2.0 adds (some) support for C++23, there is a file stdatomic.h
as part of the C++ headers. Since all directories are added to the gcc call via -isystem
, a C-only built finds that header as well, and due to the preprocessor guards in there it does nothing. Digging a bit further and asking on the libstdc++ mailinglist, I found out that a C compilation is not supposed to see that header in the first place. When using only --sysroot
and dropping the -nostdinc
and -isystem
arguments, gcc actually does the right thing. So it looks like the natural fix for this is to do exactly that: use --sysroot
and drop the -nostdinc
and -isystem
arguments. Unfortunately, it's not entirely straightforward, because gcc uses stddef.h
and stdarg.h
from the toolchain archive and then those paths are made absolute ones in the .d file, making Bazel complain about undeclared includes.
The easiest workaround is patching the C++23 stdatomic.h
, so it does #include_next
when included from C code. However, it looks like the better solution here is to properly use --sysroot
. I'm not sure why gcc makes some paths absolute in the .d file, but I can imagine that using a single archive with compiler and sysroot would make things much easier.
Why do you have the custom sysroot and explicitly remove the one from the Bootlin toolchain? Also, when you build gcc yourself as part of the sysroot, why do you even use the Bootlin toolchain?
Development (host) and target OS/architectures:
Output of bazel --version
:
bazel 5.3.1
Version of the Aspect rules, or other relevant rules from your
WORKSPACE
or MODULE.bazel
file:
patched main
Language(s) and/or frameworks involved:
C, C++
I can put up an example to repro. But understanding the rationale behind the custom sysroot would be very helpful as a start.
No response
This is to give users a stable archive whose SHA they can depend on. GitHub does not guarantee stability of the SHA over time for automatically generated archives. (Remember when they broke the world last January?)
The "righteous" thing to do here is that every time you cut a release, you should upload a .tar.gz
archive and attach it to the release. Here's an example of a release that has an artifact like this: besides the two auto-generated ones, there's also au-0.3.4.tar.gz
, whose contents are stable and will never change. Here's the section of our own release instructions that show how to obtain the .tar.gz
file easily.
Since we already compile GCC for the sysroot, we might as well just setup a separate pipeline to also publish a toolchain. It will require extra tools like binutils.
This will solve the problem we have with default RBE images that don't have libgmp, libmpc and libmpfr shared objects installed that the bootlin binaries are dynamically linked to. Since we statically link those libraries into the GCC binaries, our toolchain will play nicer with RBE.
Can I replace the current docker-based sysroots with the ones I generate using crosstools? I am looking for more control over glibc and libstdc++ versions.
.s
files.s
file to .S
, the build worksDevelopment (host) and target OS/architectures:
Output of bazel --version
: bazel 6.0.0-pre.20220909.2
Version of the Aspect rules, or other relevant rules from your
WORKSPACE
or MODULE.bazel
file:
Language(s) and/or frameworks involved:
Build rule
load("//bazel:fd_build_system.bzl", "fd_cc_library", "fd_cc_test")
package(default_visibility = ["//src/ballet:__subpackages__"])
fd_cc_library(
name = "sha512",
srcs = [
"fd_sha512.c",
"fd_sha512_core_avx2.s",
],
hdrs = [
"fd_sha512.h",
],
deps = [
"//src/ballet:base_lib",
],
)
When changing fd_sha512_core_avx2.s
to fd_sha512_core_avx2.S
, the build succeeds.
Loading: Loading: 0 packages loaded Analyzing: target //src/ballet/sha512:sha512 (0 packages loaded, 0 targets configured) INFO: Analyzed target //src/ballet/sha512:sha512 (0 packages loaded, 0 targets configured). INFO: Found 1 target... [0 / 4] [Prepa] BazelWorkspaceStatusAction stable-status.txt SUBCOMMAND: # //src/ballet/sha512:sha512 [action 'Compiling src/ballet/sha512/fd_sha512_core_avx2.s', configuration: 462f7a8353785ed4ab21eca57082c09d0458027d8309ce41bb7dd96b8ad0bff4, execution platform: @local_config_platform//:host] (cd /home/ripatel/.cache/bazel/_bazel_ripatel/c4b4531357e10c25e070592873e21602/execroot/firedancer && \ exec env - \ PATH=/bin:/usr/bin:/usr/local/bin \ PWD=/proc/self/cwd \ external/gcc11_x86-64-v2/bin/gcc -fPIC -Wno-misleading-indentation -Wno-ignored-attributes '-DFD_HAS_DOUBLE=1' '-DFD_HAS_ALLOCA=1' '-DFD_HAS_X86=1' '-D_XOPEN_SOURCE=700' '-DFD_HAS_HOSTED=1' -pthread '-DFD_HAS_THREADS=1' '-DFD_HAS_ATOMIC=1' -no-canonical-prefixes -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -isystemexternal/sysroot_x86_64//include/c++/10.3.0 -isystemexternal/sysroot_x86_64//include/c++/10.3.0/x86_64-linux -isystemexternal/sysroot_x86_64//lib/gcc/x86_64-linux/10.3.0/include-fixed -isystemexternal/sysroot_x86_64//lib/gcc/x86_64-linux/10.3.0/include -isystemexternal/sysroot_x86_64//usr/include -c src/ballet/sha512/fd_sha512_core_avx2.s -o bazel-out/k8-fastbuild/bin/src/ballet/sha512/_objs/sha512/fd_sha512_core_avx2.pic.o) # Configuration: 462f7a8353785ed4ab21eca57082c09d0458027d8309ce41bb7dd96b8ad0bff4 # Execution platform: @local_config_platform//:host ERROR: /data/prj/firedancer/src/ballet/sha512/BUILD:5:14: Compiling src/ballet/sha512/fd_sha512_core_avx2.s failed: (Exit 1): gcc failed: error executing command (from target //src/ballet/sha512:sha512) external/gcc11_x86-64-v2/bin/gcc -fPIC -Wno-misleading-indentation -Wno-ignored-attributes '-DFD_HAS_DOUBLE=1' '-DFD_HAS_ALLOCA=1' '-DFD_HAS_X86=1' '-D_XOPEN_SOURCE=700' '-DFD_HAS_HOSTED=1' -pthread ... (remaining 17 arguments skipped)Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
src/main/tools/linux-sandbox-pid1.cc:490: "execvp(external/gcc11_x86-64-v2/bin/gcc, 0x1149cf0)": No such file or directory
Target //src/ballet/sha512:sha512 failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.132s, Critical Path: 0.04s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully
Loading: Loading: 0 packages loaded Analyzing: target //src/ballet/sha512:sha512 (1 packages loaded, 0 targets configured) INFO: Analyzed target //src/ballet/sha512:sha512 (1 packages loaded, 4 targets configured). INFO: Found 1 target... [0 / 4] [Prepa] BazelWorkspaceStatusAction stable-status.txt SUBCOMMAND: # //src/ballet/sha512:sha512 [action 'Compiling src/ballet/sha512/fd_sha512_core_avx2.S', configuration: 462f7a8353785ed4ab21eca57082c09d0458027d8309ce41bb7dd96b8ad0bff4, execution platform: @local_config_platform//:host] (cd /home/ripatel/.cache/bazel/_bazel_ripatel/c4b4531357e10c25e070592873e21602/execroot/firedancer && \ exec env - \ PATH=/bin:/usr/bin:/usr/local/bin \ PWD=/proc/self/cwd \ external/gcc11_x86-64-v2/bin/gcc -MD -MF bazel-out/k8-fastbuild/bin/src/ballet/sha512/_objs/sha512/fd_sha512_core_avx2.pic.d -fPIC -iquote . -iquote bazel-out/k8-fastbuild/bin -isystem bazel-out/k8-fastbuild/bin/external/numa/libnuma_headers/include -isystem bazel-out/k8-fastbuild/bin/external/numa/libnuma/include -Wno-misleading-indentation -Wno-ignored-attributes '-DFD_HAS_DOUBLE=1' '-DFD_HAS_ALLOCA=1' '-DFD_HAS_X86=1' '-D_XOPEN_SOURCE=700' '-DFD_HAS_HOSTED=1' -pthread '-DFD_HAS_THREADS=1' '-DFD_HAS_ATOMIC=1' --sysroot external/sysroot_x86_64 -no-canonical-prefixes -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -isystemexternal/sysroot_x86_64//include/c++/10.3.0 -isystemexternal/sysroot_x86_64//include/c++/10.3.0/x86_64-linux -isystemexternal/sysroot_x86_64//lib/gcc/x86_64-linux/10.3.0/include-fixed -isystemexternal/sysroot_x86_64//lib/gcc/x86_64-linux/10.3.0/include -isystemexternal/sysroot_x86_64//usr/include -c src/ballet/sha512/fd_sha512_core_avx2.S -o bazel-out/k8-fastbuild/bin/src/ballet/sha512/_objs/sha512/fd_sha512_core_avx2.pic.o) # Configuration: 462f7a8353785ed4ab21eca57082c09d0458027d8309ce41bb7dd96b8ad0bff4 # Execution platform: @local_config_platform//:host SUBCOMMAND: # //src/ballet/sha512:sha512 [action 'Linking src/ballet/sha512/libsha512.a', configuration: 462f7a8353785ed4ab21eca57082c09d0458027d8309ce41bb7dd96b8ad0bff4, execution platform: @local_config_platform//:host] (cd /home/ripatel/.cache/bazel/_bazel_ripatel/c4b4531357e10c25e070592873e21602/execroot/firedancer && \ exec env - \ PATH=/bin:/usr/bin:/usr/local/bin \ PWD=/proc/self/cwd \ external/gcc11_x86-64-v2/bin/ar rcsD bazel-out/k8-fastbuild/bin/src/ballet/sha512/libsha512.a bazel-out/k8-fastbuild/bin/src/ballet/sha512/_objs/sha512/fd_sha512.pic.o bazel-out/k8-fastbuild/bin/src/ballet/sha512/_objs/sha512/fd_sha512_core_avx2.pic.o) # Configuration: 462f7a8353785ed4ab21eca57082c09d0458027d8309ce41bb7dd96b8ad0bff4 # Execution platform: @local_config_platform//:host SUBCOMMAND: # //src/ballet/sha512:sha512 [action 'Linking src/ballet/sha512/libsha512.so', configuration: 462f7a8353785ed4ab21eca57082c09d0458027d8309ce41bb7dd96b8ad0bff4, execution platform: @local_config_platform//:host] (cd /home/ripatel/.cache/bazel/_bazel_ripatel/c4b4531357e10c25e070592873e21602/execroot/firedancer && \ exec env - \ PATH=/bin:/usr/bin:/usr/local/bin \ PWD=/proc/self/cwd \ external/gcc11_x86-64-v2/bin/gcc -shared -o bazel-out/k8-fastbuild/bin/src/ballet/sha512/libsha512.so bazel-out/k8-fastbuild/bin/src/ballet/sha512/_objs/sha512/fd_sha512.pic.o bazel-out/k8-fastbuild/bin/src/ballet/sha512/_objs/sha512/fd_sha512_core_avx2.pic.o -pthread -Wl,-S -Wl,-z,relro,-z,now -pass-exit-codes -lm -lstdc++ --sysroot external/sysroot_x86_64 -Bexternal/gcc11_x86-64-v2/bin -Bexternal/sysroot_x86_64//usr/lib -Bexternal/sysroot_x86_64//lib64 -Lexternal/sysroot_x86_64//lib64 -Lexternal/sysroot_x86_64//usr/lib -Lexternal/sysroot_x86_64//lib/gcc/x86_64-linux/10.3.0) # Configuration: 462f7a8353785ed4ab21eca57082c09d0458027d8309ce41bb7dd96b8ad0bff4 # Execution platform: @local_config_platform//:host Target //src/ballet/sha512:sha512 up-to-date: bazel-bin/src/ballet/sha512/libsha512.a bazel-bin/src/ballet/sha512/libsha512.so INFO: Elapsed time: 0.775s, Critical Path: 0.64s INFO: 4 processes: 1 internal, 3 linux-sandbox. INFO: Build completed successfully, 4 total actions INFO: Build completed successfully, 4 total actions
I've been running into an issue where I get the following error when trying to use this toolchain to build zlib:
/home/atobaccowalla/.cache/bazel/_bazel_atobaccowalla/49400f0e8b5fdbc2448f09cd587fc4d3/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: cannot find /usr/lib/libc.so.6
/home/atobaccowalla/.cache/bazel/_bazel_atobaccowalla/49400f0e8b5fdbc2448f09cd587fc4d3/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: cannot find /usr/lib/libc_nonshared.a
/home/atobaccowalla/.cache/bazel/_bazel_atobaccowalla/49400f0e8b5fdbc2448f09cd587fc4d3/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: cannot find /usr/lib/ld-linux-x86-64.so.2
None of those files exist on my machine in /usr/lib
, but they do exist in /usr/lib64
(ex /usr/lib64/libc.so.6
):
➜ find /usr -name 'libc.so.6'
/usr/lib64/libc.so.6
That being said, I'm not sure if this toolchain is (or should be) checking paths on my machine rather than only looking inside the downloaded sysroot which contains those files in the expected location:
➜ find /home/atobaccowalla/.cache/bazel/_bazel_atobaccowalla/49400f0e8b5fdbc2448f09cd587fc4d3/execroot/gcc-toolchain-sample/external/sysroot_x86_64/ -name 'libc.so.6'
/home/atobaccowalla/.cache/bazel/_bazel_atobaccowalla/49400f0e8b5fdbc2448f09cd587fc4d3/execroot/gcc-toolchain-sample/external/sysroot_x86_64/usr/lib/libc.so.6
I'm including the full output I get when trying to build here and I've also created a minimal reproduction repo here.
Hello!
I was just wondering if you had any plans on supporting macOS and/or Windows in the future?
Since switching over from the native cpp toolchain to this hermetic one we have noticed that the execution root found under bazel-out changed from k8-fastbuild
to darwin_arm64-fastbuild
on local macos machines. This may not seem like an issue at the surface, but we also have CI which runs Bazel and in that case the host is still k8.
The effect of this is that CI & local runs no longer share cache hits because the pathing changes are included in cache keys. Additionally our remote workers see a change in the JDK from this pathing and that change causes workers to restart for no reason leading to churn and delays.
We suspect the root cause of this is from needing to remove --cpu=k8 --host_cpu=k8
when switching over to the new cpp toolchain. These flags appear to also impact the bazel-out pathing. Setting these flags with the new toolchain produces this error:
/private/var/tmp/_bazel_ted_tenedorio/544c8a90b96ef2c7562b61f1f903de27/external/local_config_cc/BUILD:28:19: in cc_toolchain_suite rule @local_config_cc//:toolchain: cc_toolchain_suite '@local_config_cc//:toolchain' does not contain a toolchain for cpu 'k8'.
Ideally we could make changes to this toolchain so that those flags can be used again.
Development (host) and target OS/architectures:
Host: Macos - cpu = darwin_arm64
Target: Linux - cpu = k8
Output of bazel --version
: 6.0.0
Version of the Aspect rules, or other relevant rules from your
WORKSPACE
or MODULE.bazel
file:
GCC Toolchain: aspect-build/gcc-toolchain/archive/refs/tags/0.4.2.tar.gz
rules_java: bazelbuild/rules_java/releases/download/0.1.1/rules_java-0.1.1.tar.gz
rules_kotlin: bazelbuild/rules_kotlin/releases/download/v1.7.1/rules_kotlin_release.tgz
JDK 11:
https://corretto.aws/downloads/resources/8.275.01.1/amazon-corretto-8.275.01.1-linux-x64.tar.gz
https://corretto.aws/downloads/resources/8.275.01.1/amazon-corretto-8.275.01.1-macosx-x64.tar.gz
Language(s) and/or frameworks involved:
CPP & Java/Kotlin. The issue is impacting all of caching though as it changes the bazel-out
relative pathing.
Run this on a macOS machine to see the bazel-out pathing of concern:
`bazel aquery //path/to/some/java_library/package --config remote`
And observe the `--output` field for the sourcejar:
action 'Building source jar path/to/some/java_library/package/package_lib-sources.jar'
Mnemonic: JavaSourceJar
Target: //path/to/some/java_library/package:package_lib
Configuration: darwin_arm64-fastbuild
Execution platform: @aspect_gcc_toolchain//platforms:x86_64_linux_remote
ActionKey: 29ca9f15b7392f2feddf3804b63bae3341fa053f43482c2384bf630347b62682
Inputs: [path/to/some/java_library/package/PackageSourceFile.kt, external/remote_java_tools_linux/java_tools/src/tools/singlejar/singlejar_local]
Outputs: [bazel-out/darwin_arm64-fastbuild/bin/path/to/some/java_library/package/package_lib-sources.jar]
ExecutionInfo: {OSFamily: Linux, container-image: REDACTED}
Command Line: (exec external/remote_java_tools_linux/java_tools/src/tools/singlejar/singlejar_local \
--output \
bazel-out/darwin_arm64-fastbuild/bin/bazel/path/to/some/java_library/package/package_lib-sources.jar \
--compression \
--normalize \
--exclude_build_data \
--warn_duplicate_resources \
--resources \
Notice how darwin_arm64-fastbuild
is in the root:
bazel-out/darwin_arm64-fastbuild/bin/bazel/path/to/some/java_library/package/package_lib-sources.jar
Running the same command on a Linux machine produces:
bazel-out/k8-fastbuild/bin/bazel/path/to/some/java_library/package/package_lib-sources.jar
You can also run this to see the cpu
flag error:
bazel build //path/to/some/java_library/package --config remote --cpu=k8 --host_cpu=k8
The remote
config here is essentially:
--java_language_version=8
--java_runtime_version=remotejdk_11
--tool_java_language_version=11
--tool_java_runtime_version=remotejdk_11
--action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1
--incompatible_strict_action_env=true
--incompatible_enable_cc_toolchain_resolution
--extra_execution_platforms=@aspect_gcc_toolchain//platforms:x86_64_linux_remote
--host_platform=@aspect_gcc_toolchain//platforms:x86_64_linux_remote
--symlink_prefix=dist/
--remote_timeout=3600
--jobs=1024
--remote_local_fallback=false
--compiler_warnings=off
--define=EXECUTOR=remote
--disk_cache=
--remote_download_minimal
--remote_default_exec_properties="container-image=REDACTED"
--remote_default_exec_properties="OSFamily=Linux"
--experimental_remote_mark_tool_inputs=true
--remote_default_exec_properties="dockerReuse=True"
--remote_executor=grpcs:REMOTE_URL
--remote_cache=grpcs:REMOTE_URL
### Any other information?
_No response_
While the rules make sure to link against a rather old glibc, this is not true for libstdc++, which comes as part of gcc. The default build mode links against libstdc++ dynamically. When running on a host that has a version that is too old, things break. We can of course work around this by linking statically. But if dynamic linking is not reliable from a portability perspective, IMHO it would be cleaner to drop support for dynamically linking libstdc++ completely.
Bazel can handle dynamic libraries well: put the appropriate symlink into the runfiles dir and set RPATH properly, so that the library is found at runtime. I think using that in the gcc-toolchain rules should be possible.
Development (host) and target OS/architectures:
Host: Ubuntu 22.04
Target: Debian Buster
Architecture: both x86_64
Output of bazel --version
:
bazel 5.3.1
Version of the Aspect rules, or other relevant rules from your
WORKSPACE
or MODULE.bazel
file:
Language(s) and/or frameworks involved:
# checkout branch where the C++ examples uses std::filesystem
git clone https://github.com/ahans/gcc-toolchain.git
cd gcc-toolchain
git switch use-filesystem
bazel build //examples/hello_world_cpp
# now try to run in a Debian Buster container
docker run -v $HOME:$HOME --workdir $(pwd) -ti debian:buster bazel-bin/examples/hello_world_cpp/hello_world_cpp
# results in:
# bazel-bin/examples/hello_world_cpp/hello_world_cpp: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by bazel-bin/examples/hello_world_cpp/hello_world_cpp)
# Replacing buster with bullseye makes it work. Bullseye ships with gcc 10.2, so that does support
# std::filesystem; gcc 8.x from buster does not
No response
I'm just wondering if there are any concerns about building all packages from source code when constructing the sysroot. It would be significantly faster to directly use apt-get for the required packages and then extract them from /lib and /usr/lib directories. In terms of reproducibility, we can specify the versions of these packages during the apt-get process. To support different architectures, we can use QEMU as an emulator.
It seems a WORKSPACE
snippet to set up gcc-toolchain is missing from the current documentation. It would be nice to include the WORKSPACE
snippet in documentation or the release page so that users can start using gcc-toolchain without troubles by copying the WORKSPACE
snippet into their WORKSPACE
files.
Based on this repo's WORKSPACE
, below is an example of the WORKSPACE
snippet and .bazelrc
to use gcc-toolchain v0.4.2 to build binaries for x86_64. It would be great to confirm that there is something wrong or redundant with the snippet. I mentioned this because unusual configs, --action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1
and --incompatible_enable_cc_toolchain_resolution
were required to add to .bazelrc
so that Bazel can use the C/C++ toolchain managed by gcc-toolchain. If these configs are required indeed, I think it worth noting it in the documentation because these configs not so obvious for non-Bazel experts.
WORKSPACE
:
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
# Skylib
# gcc-toolchain depends on bazel-skylib
http_archive(
name = "bazel_skylib",
sha256 = "f7be3474d42aae265405a592bb7da8e171919d74c16f082a5457840f06054728",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.2.1/bazel-skylib-1.2.1.tar.gz",
"https://github.com/bazelbuild/bazel-skylib/releases/download/1.2.1/bazel-skylib-1.2.1.tar.gz",
],
)
load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace")
bazel_skylib_workspace()
http_archive(
name = "aspect_gcc_toolchain",
sha256 = "3341394b1376fb96a87ac3ca01c582f7f18e7dc5e16e8cf40880a31dd7ac0e1e",
strip_prefix = "gcc-toolchain-0.4.2",
url = "https://github.com/aspect-build/gcc-toolchain/archive/refs/tags/0.4.2.tar.gz",
)
load("@aspect_gcc_toolchain//toolchain:defs.bzl", "ARCHS", "gcc_register_toolchain")
gcc_register_toolchain(
name = "gcc_toolchain_x86_64",
gcc_version = "10.3.0",
sysroot_variant = "x86_64",
target_arch = ARCHS.x86_64,
)
.bazelrc
:
# Required to use the C/C++ toolchain managed by gcc-toolchain
build --action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1
build --incompatible_enable_cc_toolchain_resolution
# optional, but preferred for performance as per https://github.com/aspect-build/gcc-toolchain/issues/85
build --experimental_reuse_sandbox_directories
It seems build performance needs to be improved compared to the build performance using the host compiler. I realized the issue in a x86-64 Linux VM on GCP when I built C++ libraries (e.g., protobuf, gRPC) with this toolchain. It seems the build using gcc-toolchain takes longer than when building C++ libraries using the host compiler. The reason seems to be costs for sandbox setup and deletion for every compile action, as far as I investigated. I noticed this performance issue can be mitigated by adding the --experimental_reuse_sandbox_directories
option to CLI or .bazelrc
. Since that option is experimental, I'm wondering there is any way to improve the build performance using gcc-toolchain.
For the record, below is the result of performance profiling on that VM. I measured the build time of the //examples/protobuf:hello_world_proto
target in this repo in three different settings:
--action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1
, --incompatible_strict_action_env=true
, --incompatible_enable_cc_toolchain_resolution
in .bazelrc
)--experimental_reuse_sandbox_directories
in .bazelrc
.$ uname -a
Linux ubuntu-0 5.15.0-1016-gcp #21~20.04.1-Ubuntu SMP Fri Aug 5 12:53:07 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/9/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:hsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 9.4.0-1ubuntu1~20.04.1' --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-9 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-9-Av3uEd/gcc-9-9.4.0/debian/tmp-nvptx/usr,hsa --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
$ bazel version
Build label: 5.2.0
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Jun 7 16:02:26 2022 (1654617746)
Build timestamp: 1654617746
Build timestamp as int: 1654617746
git clone https://github.com/aspect-build/gcc-toolchain.git
cd gcc-toolchain
git checkout 381975950d0909e1a1608c8a90858536562e4b1d # latest as of 2022-08-16
bazel build --profile=/tmp/prof //examples/protobuf:hello_world_proto
bazel analyze-profile /tmp/prof
The following table shows the outputs of the above bazel analyze-profile /tmp/prof
command for three settings. Here are few observations:
host compiler | gcc-toolchain | gcc-toolchain w/ reuse sandbox | |
---|---|---|---|
Total launch phase time | 1.418 s (0.68%) | 1.415 s (0.40%) | 1.308 s (0.57%) |
Total init phase time | 0.671 s (0.32%) | 0.673 s (0.19%) | 0.748 s (0.33%) |
Total target pattern evaluation phase time | 1.817 s (0.87%) | 5.008 s (1.40%) | 1.821 s (0.80%) |
Total interleaved loading-and-analysis phase time | 71.166 s (34.11%) | 107.429 s (30.10%) | 71.944 s (31.43% |
Total preparation phase time | 0.026 s (0.01%) | 0.017 s (0.00%) | 0.025 s (0.01%) |
Total execution phase time | 133.468 s (63.98%) | 242.359 s (67.90%) | 152.983 s (66.84%) |
Total finish phase time | 0.045 s (0.02%) | 0.054 s (0.02%) | 0.045 s (0.02%) |
Total run time | 208.613 s (100.00%) | 356.958 s (100.00%) | 228.876 s (100.00%) |
Analyzed collected files by bazel analyze-profile
in chrome://tracing
for each setting, following https://bazel.build/rules/performance. The following screenshots show the results (execution phase) of gcc-toolchain and gcc-toolchain w/ reuse sandbox, respectively (corrected order of the screenshots).
Observations
This move affected the stripped prefix of the automatically generated archive files, which therefore also changes the SHA. End users will need to migrate their builds by updating both the sha256
and strip_prefix
fields (and updating urls
too couldn't hurt).
Here's an example.
Hi there,
thanks for your awesome work on hermitic gcc toolchains for bazel. Its a very helpful idea to get build processes more deterministic and reliable.
As Bazel will drop the support for the old WORKSPACE architecture (see bazel roadmap ) and switch to new module design and it would simplify especially these kind of projects, it would be very nice to switch to this system ( Bazel modules ) with your project, too.
best regards,
Daniel
Some headers that are definitely present in the default toolchain's sysroot are not copied in the bazel sandbox.
For example bits/types/struct___jmp_buf_tag.h
.
See https://gist.github.com/0x2Adr1/51635421f64d6de8cbb14dbf40a27a4e for a minimal code that reproduces the issue.
exec env - \
BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 \
PATH=/bin:/usr/bin:/usr/local/bin \
PWD=/proc/self/cwd \
external/gcc_toolchain_x86_64/bin/gcc -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF bazel-out/k8-fastbuild/bin/_objs/main/main.pic.d '-frandom-seed=bazel-out/k8-fastbuild/bin/_objs/main/main.pic.o' -fPIC '-DBAZEL_CURRENT_REPOSITORY=""' -iquote . -iquote bazel-out/k8-fastbuild/bin -iquote external/bazel_tools -iquote bazel-out/k8-fastbuild/bin/external/bazel_tools --sysroot external/sysroot_x86_64 -no-canonical-prefixes -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' '-std=c++17' -isystemexternal/sysroot_x86_64//include/c++/10.3.0 -isystemexternal/sysroot_x86_64//include/c++/10.3.0/x86_64-linux -isystemexternal/sysroot_x86_64//lib/gcc/x86_64-linux/10.3.0/include-fixed -isystemexternal/sysroot_x86_64//lib/gcc/x86_64-linux/10.3.0/include -isystemexternal/sysroot_x86_64//usr/include -c main.cpp -o bazel-out/k8-fastbuild/bin/_objs/main/main.pic.o)
# Configuration: f68de507e19f0236b7b3947a923f4eddf8a133099bf7d87ad0db3cf41a440b09
# Execution platform: @local_config_platform//:host
Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
main.cpp:3:10: fatal error: bits/types/struct___jmp_buf_tag.h: No such file or directory
3 | #include <bits/types/struct___jmp_buf_tag.h>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Target //:main failed to build
INFO: Elapsed time: 0.798s, Critical Path: 0.13s
INFO: 4 processes: 4 internal.
FAILED: Build did NOT complete successfully
But this header is definitely present if I download and look at the toolchain content manually:
$ ls x86-64--glibc--stable-2021.11-5/x86_64-buildroot-linux-gnu/sysroot/usr/include/bits/types/struct___jmp_buf_tag.h
x86-64--glibc--stable-2021.11-5/x86_64-buildroot-linux-gnu/sysroot/usr/include/bits/types/struct___jmp_buf_tag.h
Development (host) and target OS/architectures: ArchLinux x86_64
Output of bazel --version
: bazel 6.0.0
Version of the Aspect rules, or other relevant rules from your
WORKSPACE
or MODULE.bazel
file: 22862ac
Language(s) and/or frameworks involved: C++
Minimal repro code: https://gist.github.com/0x2Adr1/51635421f64d6de8cbb14dbf40a27a4e
No response
Compilation failed when targeting aarch64. See below error:
SUBCOMMAND: # @com_google_absl//absl/strings:cord_internal [action 'Compiling absl/strings/internal/cord_rep_btree_reader.cc', configuration: 176b5d5df169c5e83a68b4875b22e319078ba78a34afd48f69d1f7629630226d, execution platform: @local_config_platform//:host]
(cd /home/gfrankliu/.cache/bazel/_bazel_gfrankliu/48efe75e9cdaaaf5b7f7393802f01694/execroot/com_github_grpc_grpc && \
exec env - \
BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 \
GRPC_BAZEL_RUNTIME=1 \
PATH=/home/gfrankliu/.local/bin:/home/gfrankliu/bin:/usr/local/buildtools/java/jdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/gfrankliu/go/bin \
PWD=/proc/self/cwd \
external/gcc_toolchain_aarch64/bin/gcc -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF bazel-out/k8-fastbuild/bin/external/com_google_absl/absl/strings/_objs/cord_internal/cord_rep_btree_reader.pic.d '-frandom-seed=bazel-out/k8-fastbuild/bin/external/com_google_absl/absl/strings/_objs/cord_internal/cord_rep_btree_reader.pic.o' -fPIC '-DBAZEL_CURRENT_REPOSITORY="com_google_absl"' -iquote external/com_google_absl -iquote bazel-out/k8-fastbuild/bin/external/com_google_absl -Wno-typedef-redefinition -DGRPC_BAZEL_BUILD '-std=c++14' -Wall -Wextra -Wcast-qual -Wconversion-null -Wformat-security -Wmissing-declarations -Woverlength-strings -Wpointer-arith -Wundef -Wunused-local-typedefs -Wunused-result -Wvarargs -Wvla -Wwrite-strings -DNOMINMAX --sysroot external/sysroot_aarch64 -no-canonical-prefixes -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' '-fdiagnostics-color=always' -nostdinc -nostdinc++ -Bexternal/gcc_toolchain_aarch64/bin -isystemexternal/sysroot_aarch64//aarch64-linux/include/c++/10.3.0 -isystemexternal/sysroot_aarch64//aarch64-linux/include/c++/10.3.0/aarch64-linux -isystemexternal/sysroot_aarch64//lib/gcc/aarch64-linux/10.3.0/include-fixed -isystemexternal/sysroot_aarch64//lib/gcc/aarch64-linux/10.3.0/include -isystemexternal/sysroot_aarch64//usr/include -c external/com_google_absl/absl/strings/internal/cord_rep_btree_reader.cc -o bazel-out/k8-fastbuild/bin/external/com_google_absl/absl/strings/_objs/cord_internal/cord_rep_btree_reader.pic.o)
# Configuration: 176b5d5df169c5e83a68b4875b22e319078ba78a34afd48f69d1f7629630226d
# Execution platform: @local_config_platform//:host
ERROR: /home/gfrankliu/.cache/bazel/_bazel_gfrankliu/48efe75e9cdaaaf5b7f7393802f01694/external/com_google_absl/absl/random/internal/BUILD.bazel:337:11: Compiling absl/random/internal/randen_hwaes.cc failed: (Exit 1): gcc failed: error executing command (from target @com_google_absl//absl/random/internal:randen_hwaes_impl) external/gcc_toolchain_aarch64/bin/gcc -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF ... (remaining 50 arguments skipped)
Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
aarch64-linux-gcc.br_real: error: unrecognized command-line option '-maes'
aarch64-linux-gcc.br_real: error: unrecognized command-line option '-msse4.1'
Target //:promise failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 52.470s, Critical Path: 2.08s
INFO: 146 processes: 24 internal, 122 linux-sandbox.
FAILED: Build did NOT complete successfully
Development (host) and target OS/architectures:
Output of bazel --version
:
INFO: Running bazel wrapper (see //tools/bazel for details), bazel version 6.3.2 will be used instead of system-wide bazel installation.
bazel 6.3.2
Version of the Aspect rules, or other relevant rules from your
WORKSPACE
or MODULE.bazel
file:
See below "How to reproduce" section for the WORKSPACE
Language(s) and/or frameworks involved:
cc/gprc
git clone https://github.com/grpc/grpc.git
cd grpc
cat >> WORKSPACE << EOF
# aspect-build
http_archive(
name = "aspect_gcc_toolchain",
sha256 = "3341394b1376fb96a87ac3ca01c582f7f18e7dc5e16e8cf40880a31dd7ac0e1e",
strip_prefix = "gcc-toolchain-0.4.2",
url = "https://github.com/aspect-build/gcc-toolchain/archive/refs/tags/0.4.2.tar.gz",
)
load("@aspect_gcc_toolchain//toolchain:defs.bzl", "gcc_register_toolchain", "ARCHS")
gcc_register_toolchain(
name = "gcc_toolchain_aarch64",
sysroot_variant = "aarch64",
target_arch = ARCHS.aarch64,
)
gcc_register_toolchain(
name = "gcc_toolchain_x86_64",
sysroot_variant = "x86_64",
target_arch = ARCHS.x86_64,
)
EOF
cat >> .bazelrc << EOF
build --action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1
build --incompatible_enable_cc_toolchain_resolution
build --reuse_sandbox_directories
EOF
bazel build //:promise -s --platforms=@aspect_gcc_toolchain//platforms:aarch64_linux
### Any other information?
_No response_
Tried registering toolchain with includes
to add something like /usr/include/linux
so <stddef.h>
could be found. Using the includes
parameter did include that directory for includes, but removed all the defaults.
Development (host) and target OS/architectures:
host: linux/amd64
targets: linux/amd64
, linux/aarch64
Output of bazel --version
: bazel 6.1.1
Version of the Aspect rules, or other relevant rules from your
WORKSPACE
or MODULE.bazel
file:
workspace(name = "my-code")
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
http_archive(
name = "rules_cc",
sha256 = "3d9e271e2876ba42e114c9b9bc51454e379cbf0ec9ef9d40e2ae4cec61a31b40",
strip_prefix = "rules_cc-0.0.6",
urls = ["https://github.com/bazelbuild/rules_cc/releases/download/0.0.6/rules_cc-0.0.6.tar.gz"],
)
http_archive(
name = "rules_pkg",
sha256 = "8c20f74bca25d2d442b327ae26768c02cf3c99e93fad0381f32be9aab1967675",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/rules_pkg/releases/download/0.8.1/rules_pkg-0.8.1.tar.gz",
"https://github.com/bazelbuild/rules_pkg/releases/download/0.8.1/rules_pkg-0.8.1.tar.gz",
],
)
load("@rules_pkg//:deps.bzl", "rules_pkg_dependencies")
rules_pkg_dependencies()
git_repository(
name = "aspect_gcc_toolchain",
commit = "4bd1f94536ee92b7c49673931773038d923ee86e",
remote = "https://github.com/aspect-build/gcc-toolchain",
)
#local_repository(
# name = "aspect_gcc_toolchain",
# path = "/home/kris/code/kriswuollett/gcc-toolchain",
#)
load("@aspect_gcc_toolchain//toolchain:repositories.bzl", "gcc_toolchain_dependencies")
gcc_toolchain_dependencies()
load("@aspect_gcc_toolchain//toolchain:defs.bzl", "ARCHS", "gcc_register_toolchain")
gcc_register_toolchain(
name = "gcc_toolchain_aarch64",
target_arch = ARCHS.aarch64,
includes = [
"%sysroot%/usr/include/linux",
],
)
gcc_register_toolchain(
name = "gcc_toolchain_armv7",
target_arch = ARCHS.armv7,
includes = [
"%sysroot%/usr/include/linux",
],
)
gcc_register_toolchain(
name = "gcc_toolchain_x86_64",
sysroot_variant = "x86_64-X11",
target_arch = ARCHS.x86_64,
includes = [
"%sysroot%/usr/include/linux",
],
)
http_archive(
name = "io_bazel_rules_go",
sha256 = "dd926a88a564a9246713a9c00b35315f54cbd46b31a26d5d8fb264c07045f05d",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.38.1/rules_go-v0.38.1.zip",
"https://github.com/bazelbuild/rules_go/releases/download/v0.38.1/rules_go-v0.38.1.zip",
],
)
load("@io_bazel_rules_go//go:deps.bzl", "go_download_sdk", "go_register_toolchains", "go_rules_dependencies")
go_rules_dependencies()
go_download_sdk(
name = "go_1_20_2_host",
version = "1.20.2",
)
go_download_sdk(
name = "go_1_20_2_linux_amd64",
goarch = "amd64",
goos = "linux",
version = "1.20.2",
)
go_download_sdk(
name = "go_1_20_2_linux_arm64",
goarch = "arm64",
goos = "linux",
version = "1.20.2",
)
go_register_toolchains()
http_archive(
name = "bazel_gazelle",
sha256 = "ecba0f04f96b4960a5b250c8e8eeec42281035970aa8852dda73098274d14a1d",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.29.0/bazel-gazelle-v0.29.0.tar.gz",
"https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.29.0/bazel-gazelle-v0.29.0.tar.gz",
],
)
load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies")
load("//:go.bzl", "go_dependencies")
# gazelle:repository_macro go.bzl%go_dependencies
go_dependencies()
gazelle_dependencies(go_sdk = "go_1_20_2_host")
http_archive(
name = "rules_rust",
sha256 = "b4e622a36904b5dd2d2211e40008fc473421c8b51c9efca746ab2ecf11dca08e",
urls = ["https://github.com/bazelbuild/rules_rust/releases/download/0.19.1/rules_rust-v0.19.1.tar.gz"],
)
load("@rules_rust//rust:repositories.bzl", "rules_rust_dependencies", "rust_register_toolchains")
rules_rust_dependencies()
rust_register_toolchains(
edition = "2021",
versions = [
"nightly/2023-03-24",
],
extra_target_triples = [
"aarch64-unknown-linux-gnu",
"x86_64-unknown-linux-gnu",
"wasm32-unknown-unknown",
"wasm32-wasi",
],
)
load("@rules_rust//crate_universe:repositories.bzl", "crate_universe_dependencies")
crate_universe_dependencies()
load("@rules_rust//crate_universe:defs.bzl", "crates_repository", "render_config")
load("//bazel:rust.bzl", "RUST_PACKAGES")
crates_repository(
name = "crate_index",
cargo_lockfile = "//:Cargo.lock",
generate_binaries = True,
lockfile = "//:Cargo.Bazel.lock",
packages = RUST_PACKAGES,
render_config = render_config(
default_package_name = "",
),
rust_version = "nightly/2023-03-05",
)
load("@crate_index//:defs.bzl", "crate_repositories")
crate_repositories()
load("@rules_rust//tools/rust_analyzer:deps.bzl", "rust_analyzer_dependencies")
rust_analyzer_dependencies()
http_archive(
name = "rules_oci",
patch_args = ["-p1"],
patches = ["//bazel:rules_oci_registry.patch"],
sha256 = "48642588e91e992772b94de06234da6601854fda0ee32a91ce8ef303cf5e5837",
strip_prefix = "rules_oci-0.3.7",
url = "https://github.com/bazel-contrib/rules_oci/releases/download/v0.3.7/rules_oci-v0.3.7.tar.gz",
)
load("@rules_oci//oci:dependencies.bzl", "rules_oci_dependencies")
rules_oci_dependencies()
load("@rules_oci//oci:repositories.bzl", "LATEST_CRANE_VERSION", "LATEST_ZOT_VERSION", "oci_register_toolchains")
register_toolchains("//registry:registry_toolchain")
oci_register_toolchains(
name = "oci",
crane_version = LATEST_CRANE_VERSION,
zot_version = LATEST_ZOT_VERSION,
)
load("@rules_oci//oci:pull.bzl", "oci_pull")
oci_pull(
name = "distroless_base",
digest = "sha256:5812871f5d87d6d4c226c536be70f7a8232a77230675b4b574c9866c8dc982fa",
image = "gcr.io/distroless/base",
platforms = [
"linux/amd64",
"linux/arm64",
],
)
oci_pull(
name = "distroless_cc",
digest = "sha256:f252d3ca44b7f2c718c67c2020feee7b7fd4ee6bff8a192dfea6e599b7d820ad",
# TODO(https://github.com/bazel-contrib/rules_oci/issues/74) Use latest images
# digest = "sha256:fb402c45f3ef485ccd56ca2af2a58615fc47c4978bb3004e8663a83456791f48",
image = "gcr.io/distroless/cc",
platforms = [
"linux/amd64",
"linux/arm64",
],
)
Language(s) and/or frameworks involved:
Rust
load("@aspect_gcc_toolchain//toolchain:defs.bzl", "ARCHS", "gcc_register_toolchain")
gcc_register_toolchain(
name = "gcc_toolchain_aarch64",
target_arch = ARCHS.aarch64,
includes = [
"%sysroot%/usr/include/linux",
],
)
The documentation says that parameters like includes
are supposed to be extra values:
The invocation of gcc-toolchain
uses dict::pop
that has the base values as the default
which is only used if the key
like "includes
" is not in the dict
.
So it is not possible to add extra include directories without copy and pasting the directories from the following code:
Original error without specifying includes
:
error occurred: Command "/home/kris/.cache/bazel/_bazel_kris/ebd20a16b5bb6af41a7a481997cc3984/sandbox/linux-sandbox/556/execroot/my-code/external/gcc_toolchain_aarch64/bin/gcc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-gdwarf-4" "-fno-omit-frame-pointer" "--sysroot" "external/sysroot_aarch64" "-no-canonical-prefixes" "-fno-canonical-system-headers" "-Wno-builtin-macro-redefined" "-D__DATE__=\"redacted\"" "-D__TIMESTAMP__=\"redacted\"" "-D__TIME__=\"redacted\"" "-fdiagnostics-color=always" "-nostdinc" "-Bexternal/gcc_toolchain_aarch64/bin" "-isystemexternal/sysroot_aarch64//aarch64-linux/include/c++/10.3.0" "-isystemexternal/sysroot_aarch64//aarch64-linux/include/c++/10.3.0/aarch64-linux" "-isystemexternal/sysroot_aarch64//lib/gcc/aarch64-linux/10.3.0/include-fixed" "-isystemexternal/sysroot_aarch64//lib/gcc/aarch64-linux/10.3.0/include" "-isystemexternal/sysroot_aarch64//usr/include" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-fvisibility=hidden" "-ffunction-sections" "-fdata-sections" "-fmerge-all-constants" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-o" "/home/kris/.cache/bazel/_bazel_kris/ebd20a16b5bb6af41a7a481997cc3984/sandbox/linux-sandbox/556/execroot/my-code/bazel-out/k8-fastbuild-ST-87f845362927/bin/external/crate_index__zstd-sys-2.0.7-zstd.1.5.4/zstd-sys_build_script.out_dir/zstd/lib/common/ent
Error with specifying includes
, the original base system includes are missing:
error occurred: Command "/home/kris/.cache/bazel/_bazel_kris/ebd20a16b5bb6af41a7a481997cc3984/sandbox/linux-sandbox/612/execroot/my-code/external/gcc_toolchain_aarch64/bin/gcc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-gdwarf-4" "-fno-omit-frame-pointer" "--sysroot" "external/sysroot_aarch64" "-no-canonical-prefixes" "-fno-canonical-system-headers" "-Wno-builtin-macro-redefined" "-D__DATE__=\"redacted\"" "-D__TIMESTAMP__=\"redacted\"" "-D__TIME__=\"redacted\"" "-fdiagnostics-color=always" "-nostdinc" "-Bexternal/gcc_toolchain_aarch64/bin" "-isystemexternal/sysroot_aarch64//usr/include/linux" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-fvisibility=hidden" "-ffunction-sections" "-fdata-sections" "-fmerge-all-constants" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-o" "/home/kris/.cache/bazel/_bazel_kris/ebd20a16b5bb6af41a7a481997cc3984/sandbox/linux-sandbox/612/execroot/my-code/bazel-out/k8-fastbuild-ST-87f845362927/bin/external/crate_index__zstd-sys-2.0.7-zstd.1.5.4/zstd-sys_build_script.out_dir/zstd/lib/common/entropy_common.o" "-c" "zstd/lib/common/entropy_common.c" with args "gcc" did not execute successfully (status code exit status: 1).
incompatible_sandbox_hermetic_tmp causes some relative paths to change, and results in this logic no longer pointing to the correct location (on my machine, it just ends up being /tmp)
This produces an error like:
external/gcc_toolchain_armv7/bin/gcc: line 45: /tmp/external/gcc_toolchain_armv7_files/bin/arm-linux-gcc: No such file or directory
Development (host) and target OS/architectures: linux/amd64
Output of bazel --version
: 7.0.0
Version of the Aspect rules, or other relevant rules from your
WORKSPACE
or MODULE.bazel
file:
Language(s) and/or frameworks involved:
from aspect/gcc-toolchain:
echo 7.0.0 > .bazelversion
bazel build --noenable_bzlmod //...
ERROR: $ROOT/aspect/gcc-toolchain/examples/hello_world_cpp/BUILD.bazel:19:10: Compiling examples/hello_world_cpp/main.cpp failed: (Exit 127): gcc failed: error executing CppCompile command (from target //examples/hello_world_cpp:hello_world_cpp) external/gcc_toolchain_armv7/bin/gcc -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF ... (remaining 32 arguments skipped)
Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
external/gcc_toolchain_armv7/bin/gcc: line 45: /tmp/external/gcc_toolchain_armv7_files/bin/arm-linux-gcc: No such file or directory
What seems to work is to pass the previously calculated execroot into _render_tool_paths and then further into the wrapper via a new __EXECROOT__
substitution but i'm not sure if that's an approach that would backfire in some way.
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
These updates are currently rate-limited. Click on a checkbox below to force their creation now.
These updates have all been created already. Click a checkbox below to force a retry/rebase of any.
internal.bzl
io_bazel_stardoc 0.5.6
aspect_bazel_lib v1.33.0
rules_foreign_cc 0.9.0
lapack v3.11.0
com_google_protobuf v3.21.12
rules_pkg 0.9.1
toolchain/repositories.bzl
bazel_skylib 1.4.2
aspect_bazel_lib v1.33.0
rules_cc 0.0.8
.bazelversion
bazel 6.3.1
sysroot/Dockerfile
ubuntu 22.04
.github/workflows/default.yaml
actions/checkout v4
bazelbuild/setup-bazelisk v2
actions/checkout v4
bazelbuild/setup-bazelisk v2
actions/checkout v4
bazelbuild/setup-bazelisk v2
actions/checkout v4
bazelbuild/setup-bazelisk v2
actions/checkout v4
bazelbuild/setup-bazelisk v2
ubuntu 20.04
ubuntu 20.04
ubuntu 20.04
.github/workflows/new_issue.yaml
actions/github-script v6
Currently it's a mystery what is the value prop, and how to adopt and use this repo.
rules_js and rules_py READMEs are a place to start - and we need high-level documentation as well.
First of all: Thanks for the great work of enabling an hermetic gcc toolchain in Bazel! It helped us already a lot for our understanding.
At the moment we're trying to integrate your toolchain (with a bumped gcc
version) in our build but I am already failing to compile a third party project (which we're bundling/shipping/linking against).
The configure_make
step is failing during configure
with a lot of errors, where some of them seem to be toolchain related. e.g.:
configure:3724: /home/timotheus/.cache/bazel/_bazel_timotheus/17bdf0994d16870d211e03bc3a9d26dd/sandbox/linux-sandbox/21/execroot/_main/external/gcc_toolchain_x86_64/bin/gcc -c -Dredacted="redacted" conftest.c >&5
conftest.c:10:10: fatal error: stdio.h: No such file or directory
10 | #include <stdio.h>
| ^~~~~~~~~
compilation terminated.
This looks to me as configure
is expecting gcc
to find the system include paths by its own.
But those paths are explicitly registered (as I assume the pre-compiled/relocated gcc
won't be able to do that on his own)
gcc_toolchain_workspace_name = rctx.attr.gcc_toolchain_workspace_name,
target_compatible_with = str(target_compatible_with),
toolchain_files_repository_name = rctx.attr.toolchain_files_repository_name,
# Sysroot
sysroot = sysroot,
# Includes
cxx_builtin_include_directories = str(cxx_builtin_include_directories),
This may all be an issue of the external rrdtool
project but you guys may have more insights / tips regarding that?
The issue should be fully reproducible with: Checkmk@3be3a14
bazel build --sandbox_add_mount_pair=/tmp --sandbox_debug --verbose_failures --platforms=//platforms:x86_64_linux //examples/rrdtool:rrdtool
We are adding a large set of warning flags with extra_cxxflags
and extra_cflags
. I just tried to disable a warning locally in a cc_library
with with copts = ["-Wno-foo"]
, and it surprisingly didn't work. I built with bazel build -s
and saw that the compiler invocation adds the extra_cxxflags
after the flags from copts
. This means that one cannot disable any flag from extra_cxxflags
locally in a cc_library
.
Was it intentional to do it this way or was that just how it was set up?
If it makes sense, then the feature would be to swap the order of extra_cxxflags
and copts
. (Same for extra_cflags
).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.