Giter Site home page Giter Site logo

cgrindel / rules_spm Goto Github PK

View Code? Open in Web Editor NEW
59.0 5.0 13.0 358 KB

Provide a means for integrating external Swift packages built by Swift Package Manager into Bazel build using rules_swift.

License: Apache License 2.0

Starlark 96.06% Smarty 0.35% Shell 1.14% Swift 2.46%
swift bazel bazel-rules swift-package-manager

rules_spm's Introduction

Swift Package Manager Rules for Bazel

swift_bazel is now rules_swift_package_manager

2023-04-03: The swift_bazel repository was renamed to rules_swift_package_manager. The successor to rules_spm now supersedes the functionality in this repository. As a result, this repository is being archived.

swift_bazel is replacing rules_spm

2023-01-10: The swift_bazel project, the successor for rules_spm, is now ready for testing. In addition to managing external Swift package dependencies, it also provides functionality for generating Bazel build files for your Swift project. If you have used rules_spm for your projects or if you are just looking for a way to manage external Swift packages in your Bazel workspace, I encourage you to check out swift_bazel and provide your feedback.

Important Message for iOS Developers

2022-11-28: There are some known issues with building and deplying applications for iOS devices using rules_spm. If you need to deploy to an iOS device, we do not recommend using rules_spm, at this time.

Work is currently underway to implement a Gazelle plugin that will resolve external package dependencies and generate Bazel build files that will build the external packages using rules_swift. This will allow applications to be built and deployed to any platforms/devices supported by rules_swift and rules_apple. Information about the plugin will be posted to here and bazelbuild.slack.com in the coming weeks.

Build

This repository contains rules for Bazel that can be used to download, build and consume Swift packages with rules_swift rules. The rules in this repository build the external Swift packages with Swift Package Manager, then make the outputs available to Bazel rules.

Table of Contents

Reference Documentation

Click here for reference documentation for the rules and other definitions in this repository.

Prerequisites

Mac OS

Be sure to install Xcode.

Linux

You will need to install Swift. Make sure that running swift --version works properly.

Don't forget that rules_swift expects the use of clang. Hence, you will need to specify CC=clang before running Bazel.

In addition, the Bazel toolchains want to use the Gold linker. While it may be installed on the system, it is possible that it will not be findable when using clang as mentioned previously. So, you will want to do one of the following:

Option #1: Create a symlink to the linker in the clang directory.

sudo ln -s $(which ld.gold) $(dirname $(which clang))

This option worked well on a fairly minimal install of Ubuntu 20.04.

Option #2: Specify a custom PATH via --action_env

Specify a custom PATH to Bazel via --action_env. The custom PATH should have the Swift bin directory as the first item.

swift_exec=$(which swift)
real_swift_exec=$(realpath $swift_exec)
real_swift_dir=$(dirname $real_swift_exec)
new_path="${real_swift_dir}:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
cat >>local.bazelrc <<EOF
build --action_env=PATH=${new_path}
EOF

This approach was necessary to successfully execute the examples on an Ubuntu runner using Github actions. See the Github workflow for more details.

Quickstart

The following provides a quick introduction on how to use the rules in this repository. Also, check out the examples directory for more information.

1. Configure your workspace to use rules_spm

Add the following to your WORKSPACE file to add this repository and its dependencies.

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "cgrindel_rules_spm",
    sha256 = "777e687245faa7340488e61f5abb23b95b4c0e27e05f7cea7318c03e4cc38289",
    strip_prefix = "rules_spm-0.11.2",
    urls = [
        "http://github.com/cgrindel/rules_spm/archive/v0.11.2.tar.gz",
    ],
)

load(
    "@cgrindel_rules_spm//spm:deps.bzl",
    "spm_rules_dependencies",
)

spm_rules_dependencies()

load(
    "@build_bazel_rules_swift//swift:repositories.bzl",
    "swift_rules_dependencies",
)

swift_rules_dependencies()

load(
    "@build_bazel_rules_swift//swift:extras.bzl",
    "swift_rules_extra_dependencies",
)

swift_rules_extra_dependencies()

2. Add external Swift packages as dependencies to your workspace

Add the following to download and build apple/swift-log to your workspace. NOTE: It is imperative that the products that will be used by your project are listed in the products list.

load("@cgrindel_rules_spm//spm:defs.bzl", "spm_pkg", "spm_repositories")

spm_repositories(
    name = "swift_pkgs",
    dependencies = [
        spm_pkg(
            "https://github.com/apple/swift-log.git",
            exact_version = "1.4.2",
            products = ["Logging"],
        ),
    ],
)

3. Use the module(s) defined in the Swift packages

Each Swift package can contain multiple Swift modules. A Bazel target is created for each Swift module which is exported by the products that were listed in the spm_pkg declaration.

The following shows a Bazel BUILD file with a swift_binary that depends upon the Logging module defined in apple/swift-log.

load("@build_bazel_rules_swift//swift:swift.bzl", "swift_binary")

swift_binary(
    name = "simple",
    srcs = ["main.swift"],
    visibility = ["//swift:__subpackages__"],
    deps = [
        "@swift_pkgs//swift-log:Logging",
    ],
)

Lastly, import the Swift module and enjoy!

import Logging

let logger = Logger(label: "com.example.main")
logger.info("Hello World!")

rules_spm's People

Contributors

cgrindel avatar cgrindel-app-token-generator[bot] avatar pomozoff avatar pswaminathan avatar renovate[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

rules_spm's Issues

Fix failing integration test "Test Simple Example, Build with Older Xcode"

Description

The test started failing in CI around Nov 3, 2021. It works on my dev machine (macOS). For now, the test has been disabled.

Error

 error: terminated(128): DEVELOPER_DIR=/Applications/Xcode_12.4.app/Contents/Developer __CF_USER_TEXT_ENCODING=0x1F5:0:0 TMPDIR=/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/ _=bazel-out/host/bin/external/build_bazel_rules_swift/tools/worker/worker GIT_SSH_COMMAND=ssh -oBatchMode=yes *** XCODE_VERSION_OVERRIDE=12.4.0.12D4e SDKROOT=/Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk SHLVL=1 PATH=/private/var/tmp/_bazel_runner/4bd0d8640a6b3aa42285e35d21688dd6/sandbox/darwin-sandbox/204/execroot/simple_example/external/cgrindel_rules_spm_local_config/spm_utilities:/usr/gnu/bin:/usr/local/bin:/bin:/usr/bin:. GIT_TERMINAL_PROMPT=0 git -C /private/var/tmp/_bazel_runner/4bd0d8640a6b3aa42285e35d21688dd6/sandbox/darwin-sandbox/204/execroot/simple_example/bazel-out/darwin-fastbuild/bin/external/swift_pkgs/spm_build/repositories/swift-log-df1c07b5 remote update -p output:
 unable to restore state from /private/var/tmp/_bazel_runner/4bd0d8640a6b3aa42285e35d21688dd6/sandbox/darwin-sandbox/204/execroot/simple_example/bazel-out/darwin-fastbuild/bin/external/swift_pkgs/spm_build/workspace-state.json; missingKey("path")
     fatal: not a git repository (or any of the parent directories): .git

Initial Thoughts

  • I think that it might be related to this SPM issue: apple/swift-package-manager@f2dd6ce
  • Actually, it looks like this is happening because a git operation is trying to happen during a compile. This should not happen. Instead, the fetch should have retrieved the repository during the loading phase.
  • If the problem is that the version of SPM that fetched the repo is different from the one compiling, which can happen because the fetch happens in loading which is a sibling operation to the toolchain resolution, then we need to figure out a way to use the correct version. The question is why does this not fail on my dev machine?

A clang target must have at least one public header

A lot of issues appear when I try to build this package:

        spm_pkg(
            url = "https://github.com/SDWebImage/libwebp-Xcode.git",
            from_version = "1.2.1",
            products = ["libwebp"],
        ),
  1. It has non-standard naming that doesn't match its URL, it leads to errors when rules_spm fails to find the package
  2. A clang target must have at least one public header. target: libwebp error appears after fixing the first issue

Select swift version

๐Ÿ‘‹

When having multiple installed xcodes, how to select the swift version without using xcode-select?

I think this can currently be done by passing the DEVELOPER_DIR variable to bazel. But this is problematic because you cant forget to set it before running bazel. A better solution would be to be able to specify it at the moment of the declaration of the spm_repositories:

    spm_repositories(
        name = "spm_" + name,
        platforms = [
            ".macOS(.v10_15)",
        ],
        dependencies = [
            ...
        ],
        env = {
            "DEVELOPER_DIR" : "path/to/xcode.app",
        },
    )

What do you think?

Ensure rules_spm work on Linux

Tasks

  • Get it working on Linux
  • Update the README.md to include Linux-specifc install directions. Specifically, they will need to add a symlink to ld.gold to their clang dir.
  • Add Linux build to Github actions workflow

Support building SPM packages located locally in the repository

Hey ๐Ÿ‘‹

Thanks for this! โญ Was wondering if it is possible to build SPM packages located locally in the repository.

Why would somebody prefer to build a local SPM package with the rules_spm instead of the rules_swift?

  • rules_spm uses SPM for building and does not require creation of BUILD.bazel files
  • Because when you have a Package.swift file, you also get a 100% working xcode project for free. If you use rules_swift and BUILD.bazel files, it is not that clear how to open an xcode project to work on the code

Progress

I have tried this before myself: I created a repository rule to build the swift package with SPM and expose the built outputs to bazel. But so far I figured out this is not so easy due to the lack of support for glob in repository rules. Without glob it is not possible to specify the sources of the package, which are required for building

the library 'Placeholder' requires macos 10.10, but depends on the product 'PostgresKit' which requires macos 10.15

Hi ๐Ÿ˜„

When having PostgresKit as a dependency I am hitting this error: error: the library 'Placeholder' requires macos 10.10, but depends on the product 'PostgresKit' which requires macos 10.15; consider changing the library 'Placeholder' to require macos 10.15 or later, or the product 'PostgresKit' to require macos 10.10 or earlier.

This is my WORKSPACE file:

http_archive(
    name = "cgrindel_rules_spm",
    sha256 = "e45ae4d99ec0f8505cb8663368d8574a3eea481e8e966787fd02aabaf793007a",
    strip_prefix = "rules_spm-0.4.0",
    url = "https://github.com/cgrindel/rules_spm/archive/v0.4.0.tar.gz",
)

load("@cgrindel_rules_spm//spm:spm.bzl", "spm_pkg", "spm_repositories")

spm_repositories(
    name = "swift_pkgs",
    dependencies = [
        spm_pkg(
            "https://github.com/vapor/postgres-kit.git",
            from_version = "2.3.2",
            products = ["PostgresKit"],
        ),
    ],
)

Seems like the Placeholder needs to be specific about its supported platform, similar as in here

Update toolchain evaluation to use Bazel toolchain evaluation

Related TODO comment in rules_spm.

The toolchain evaluation in rules_spm mimics the logic that rules_swift uses. In short, the spm_autoconfiguration repository rule executes a crude determination of whether to use Xcode or a standard Swift toolchain. Ideally, this would be evaluated using Bazel toolchains. The problem is that Bazel toolchains are evaluated once at build invocation. However, some of the Apple-specific rules (e.g. ios_unit_test) influence the toolchain that needs to be used for a build.

Goal: Update the toolchain evaluation to work with Bazel toolchains/platform evaluation and handle the Apple rule toolchain nuances.

Make sure that build for other platforms works

Even though the --arch was being passed to swift build, it turns out that is not sufficient. In short, one needs to do something like this:

swift build -v -Xswiftc "-sdk" -Xswiftc "`xcrun --sdk iphonesimulator --show-sdk-path`" -Xswiftc "-target" -Xswiftc "x86_64-apple-ios13.0-simulator"

Need to update the toolchains and build script to accommodate the information collection and build.

Make the rules_spm repo public

Tasks

  • Add license file
  • Update README with usage info
  • Add CI badge to README
  • Create an initial release and update README with the correct tag and SHA256

Revert use of custom Bazel for macos_build in CI workflow

Remove the use of the custom Bazel in themacos_build.

    # Remove this when Keith's PR is merged and made available in a Bazel release.
    env:
      USE_BAZEL_VERSION: cgrindel/6.0.0-keith_patch

Background

Error from GitHub Actions run.

ERROR: /private/var/tmp/_bazel_runner/6ba398470370e09a654e1f7bdb97a6ab/external/io_bazel_stardoc/stardoc/BUILD:70:12: Target '@io_bazel_stardoc//stardoc:renderer' depends on toolchain '@local_config_cc//:cc-compiler-darwin', which cannot be found: error loading package '@local_config_cc//': cannot load '@local_config_cc_toolchains//:osx_archs.bzl': no such file'

Keith's PR with fix.

Chuck's custom Bazel release with Keith's patch

Related Issues:

Platform mismatch when specifying platforms not including the host

๐Ÿ‘‹ Hiya @cgrindel! Thanks a ton for this project, it's been really interesting to learn how we can use SwiftPM to build products for Bazel to consume. I'm having a bit of a tough time understanding where I'm going wrong getting a target to build for iOS only using rules_spm.

Repro

I put together a small example module in a fork that illustrates the bug I'm running into.

  1. Add platforms = [".iOS(.v14)"] to the spm_repositories invocation in the WORKSPACE
  2. Add a Swift package that includes platforms within it's package description, like apollographql/apollo-ios
  3. Build the module with bazel build //Sources:Foo

Expected Behavior

The bazel build invocation builds the SwiftPM target only for the given platforms (in this case, iOS), since there's no need to build dependent modules for other platforms.

Actual Behavior

The following error is thrown from the SwiftPM invocation

error: the library 'Placeholder' requires macos 10.10, but depends on the product 'Apollo' which requires macos 10.14; consider changing the library 'Placeholder' to require macos 10.14 or later, or the product 'Apollo' to require macos 10.10 or earlier.
Verbose output
โฏ bazel build //Sources:Foo --sandbox_debug
INFO: Analyzed target //Sources:Foo (24 packages loaded, 1765 targets configured).
INFO: Found 1 target...
ERROR: /private/var/tmp/_bazel_eliperkins/0e5e28979b49af89fb8d84a6e03e62ba/external/swift_pkgs/BUILD.bazel:644:12: Building Swift package (external/swift_pkgs) for x86_64-apple-macosx12.1 using SPM. failed: (Exit 1): sandbox-exec failed: error executing command
  (cd /private/var/tmp/_bazel_eliperkins/0e5e28979b49af89fb8d84a6e03e62ba/sandbox/darwin-sandbox/269/execroot/ios_app && \
  exec env - \
    DEVELOPER_DIR=/Applications/Xcode-13.2.1.app/Contents/Developer \
    TMPDIR=/var/folders/v0/4yqzqdxd6j13p6kl5c8qwlhh0000gn/T/ \
    XCODE_VERSION_OVERRIDE=13.2.1.13C100 \
  /usr/bin/sandbox-exec -f /private/var/tmp/_bazel_eliperkins/0e5e28979b49af89fb8d84a6e03e62ba/sandbox/darwin-sandbox/269/sandbox.sb /var/tmp/_bazel_eliperkins/install/0c7899cb691a00c6ca493ede5765e1af/process-wrapper '--timeout=0' '--kill_delay=15' bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/cgrindel_rules_spm/spm/private/exec_spm_build --worker bazel-out/host/bin/external/build_bazel_rules_swift/tools/worker/worker --sdk_name macosx --spm_utility external/cgrindel_rules_spm_local_config/spm_utilities/git -Xspm --disable-sandbox -Xspm --configuration -Xspm release -Xspm --manifest-cache -Xspm none -Xspm --disable-repository-cache -Xswiftc -target -Xswiftc x86_64-apple-macosx12.1 -Xcc -target -Xcc x86_64-apple-macosx12.1 --package-path external/swift_pkgs --build-path bazel-out/darwin-fastbuild/bin/external/swift_pkgs/spm_build external/swift_pkgs/spm_build/checkouts/SQLite.swift/Sources/SQLiteObjc/include/SQLiteObjc.h bazel-out/darwin-fastbuild/bin/external/swift_pkgs/spm_build/x86_64-apple-macosx/release/SQLiteObjc.build/SQLiteObjc.h bazel-out/darwin-fastbuild/bin/external/swift_pkgs/modulemaps/SQLiteObjc.build/module.modulemap bazel-out/darwin-fastbuild/bin/external/swift_pkgs/spm_build/x86_64-apple-macosx/release/SQLiteObjc.build/module.modulemap)
error: the library 'Placeholder' requires macos 10.10, but depends on the product 'Apollo' which requires macos 10.14; consider changing the library 'Placeholder' to require macos 10.14 or later, or the product 'Apollo' to require macos 10.10 or earlier.
Target //Sources:Foo failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /Users/eliperkins/src/rules_spm/examples/ios_app/Sources/BUILD.bazel:3:14 Compiling Swift module //Sources:Foo failed: (Exit 1): sandbox-exec failed: error executing command
  (cd /private/var/tmp/_bazel_eliperkins/0e5e28979b49af89fb8d84a6e03e62ba/sandbox/darwin-sandbox/269/execroot/ios_app && \
  exec env - \
    DEVELOPER_DIR=/Applications/Xcode-13.2.1.app/Contents/Developer \
    TMPDIR=/var/folders/v0/4yqzqdxd6j13p6kl5c8qwlhh0000gn/T/ \
    XCODE_VERSION_OVERRIDE=13.2.1.13C100 \
  /usr/bin/sandbox-exec -f /private/var/tmp/_bazel_eliperkins/0e5e28979b49af89fb8d84a6e03e62ba/sandbox/darwin-sandbox/269/sandbox.sb /var/tmp/_bazel_eliperkins/install/0c7899cb691a00c6ca493ede5765e1af/process-wrapper '--timeout=0' '--kill_delay=15' bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/cgrindel_rules_spm/spm/private/exec_spm_build --worker bazel-out/host/bin/external/build_bazel_rules_swift/tools/worker/worker --sdk_name macosx --spm_utility external/cgrindel_rules_spm_local_config/spm_utilities/git -Xspm --disable-sandbox -Xspm --configuration -Xspm release -Xspm --manifest-cache -Xspm none -Xspm --disable-repository-cache -Xswiftc -target -Xswiftc x86_64-apple-macosx12.1 -Xcc -target -Xcc x86_64-apple-macosx12.1 --package-path external/swift_pkgs --build-path bazel-out/darwin-fastbuild/bin/external/swift_pkgs/spm_build external/swift_pkgs/spm_build/checkouts/SQLite.swift/Sources/SQLiteObjc/include/SQLiteObjc.h bazel-out/darwin-fastbuild/bin/external/swift_pkgs/spm_build/x86_64-apple-macosx/release/SQLiteObjc.build/SQLiteObjc.h bazel-out/darwin-fastbuild/bin/external/swift_pkgs/modulemaps/SQLiteObjc.build/module.modulemap bazel-out/darwin-fastbuild/bin/external/swift_pkgs/spm_build/x86_64-apple-macosx/release/SQLiteObjc.build/module.modulemap)
INFO: Elapsed time: 17.604s, Critical Path: 4.49s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully

From here, my next step was to add .macOS(.v10_15) to work around this issue, which works! I'm not sure if there's something funky going on here that's causing the SwiftPM invocation to build a target which assumes the host platform or something. I'm relatively new to Bazel so forgive me if I'm missing something here!

Handle project that uses rules_spm and one of its dependencies also uses rules_spm

Project A - Has dep on a package (e.g. swift-nio).
Project B - Has dep on Project A and a package that Project A uses as a dep (e.g. swift-nio)

Problem:

  1. If spm_repositories for Project A and B have the same name, we will get a duplicate declaration.
  2. If spm_repositories for Project A and B have the same names, we will get duplicate symbol errors during compile/linking

Options:

  1. Require that Project A provides a Package.swift; Project B would use spm_repositories to use the dependency.
  2. Rework rules_spm deps logic;
  • Each project that uses rules_spm would provide their package info (i.e., deps and platforms) via function.
  • The root project would collect all of the package info for all of the deps (and deps of deps) and pass it to spm_repositories.

Option 1 Thoughts

  • Biggest benefit: SPM figures it all out; it already handles all of the dep merging mechanics
  • We could provide a build rule to generate a Package.swift; it could be copied to source using rules_updatesrc.
  • Non-Bazel projects can consume the project
  • Biggest downside: Can't treat the dep repository as a Bazel repo (i.e., can't use local_repository)
    • Note: spm_pkg does support local packages. So, this may not be a big deal.

Option 2 Thoughts:

  • Biggest benefit: Can treat dep repository as a Bazel repo
  • Biggest downside: Need to rework package declarations
    • Each project that uses rules_spm would implement a function that returns their package info (deps and platforms)

Address "clang: warning: using sysroot for 'MacOSX' but targeting 'iPhone' [-Wincompatible-sysroot]"

Still seeing clang warnings when building examples/ios_sim.

INFO: From Building Swift package (external/swift_pkgs) for x86_64-apple-ios14.0-simulator using SPM.:
clang: warning: using sysroot for 'MacOSX' but targeting 'iPhone' [-Wincompatible-sysroot]
clang: warning: using sysroot for 'MacOSX' but targeting 'iPhone' [-Wincompatible-sysroot]
clang: warning: using sysroot for 'MacOSX' but targeting 'iPhone' [-Wincompatible-sysroot]
clang: warning: using sysroot for 'MacOSX' but targeting 'iPhone' [-Wincompatible-sysroot]
clang: warning: using sysroot for 'MacOSX' but targeting 'iPhone' [-Wincompatible-sysroot]
clang: warning: using sysroot for 'MacOSX' but targeting 'iPhone' [-Wincompatible-sysroot]
clang: warning: using sysroot for 'MacOSX' but targeting 'iPhone' [-Wincompatible-sysroot]
[1/8] Compiling CNIOWindows WSAStartup.c
[2/8] Compiling CNIOLinux liburing_shims.c
[3/8] Compiling CNIOWindows shim.c
[4/8] Compiling CNIOLinux shim.c
[5/8] Compiling CNIODarwin shim.c
[6/8] Compiling _NIODataStructures Heap.swift
[7/8] Compiling c-nioatomics.c
[8/8] Compiling c-atomics.c
[9/9] Compiling NIOConcurrencyHelpers NIOAtomic.swift
[10/10] Compiling NIOCore AddressedEnvelope.swift
[11/12] Compiling NIOEmbedded Embedded.swift
[12/12] Compiling NIOPosix BSDSocketAPICommon.swift
[13/13] Compiling NIO Exports.swift
[14/14] Compiling Placeholder Placeholder.swift
[14/14] Build complete!

When one adds -v to the swift build, you can see that the clang commands are still using the macosx target triple instead of the iphone simulator target triple.

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -fobjc-arc -target x86_64-apple-macosx10.10 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk -F /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks -fPIC -g -O2 -DSWIFT_PACKAGE=1 -fblocks -fmodules -fmodule-name=CNIOLinux -I /private/var/tmp/_bazel_chuck/b9ed0dda2ab004959d6341817e7c1001/sandbox/darwin-sandbox/1546/execroot/ios_simulator_example/bazel-out/apl-darwin_x86_64-fastbuild/bin/external/swift_pkgs/spm_build/checkouts/swift-nio/Sources/CNIOLinux/include -fmodules-cache-path=/private/var/tmp/_bazel_chuck/b9ed0dda2ab004959d6341817e7c1001/sandbox/darwin-sandbox/1546/execroot/ios_simulator_example/bazel-out/apl-darwin_x86_64-fastbuild/bin/external/swift_pkgs/spm_build/x86_64-apple-macosx/release/ModuleCache -target x86_64-apple-macosx11.3 -MD -MT dependencies -MF /private/var/tmp/_bazel_chuck/b9ed0dda2ab004959d6341817e7c1001/sandbox/darwin-sandbox/1546/execroot/ios_simulator_example/bazel-out/apl-darwin_x86_64-fastbuild/bin/external/swift_pkgs/spm_build/x86_64-apple-macosx/release/CNIOLinux.build/liburing_shims.c.d -c /private/var/tmp/_bazel_chuck/b9ed0dda2ab004959d6341817e7c1001/sandbox/darwin-sandbox/1546/execroot/ios_simulator_example/bazel-out/apl-darwin_x86_64-fastbuild/bin/external/swift_pkgs/spm_build/checkouts/swift-nio/Sources/CNIOLinux/liburing_shims.c -o /private/var/tmp/_bazel_chuck/b9ed0dda2ab004959d6341817e7c1001/sandbox/darwin-sandbox/1546/execroot/ios_simulator_example/bazel-out/apl-darwin_x86_64-fastbuild/bin/external/swift_pkgs/spm_build/x86_64-apple-macosx/release/CNIOLinux.build/liburing_shims.c.o

As of the writing of this issue (2021-08-26), the exec_spm_build.sh script executes the SPM build as follows:

xcrun \
  swift build \
  --manifest-cache none \
  --disable-sandbox \
  --disable-repository-cache \
  --configuration ${build_config} \
  --package-path "${package_path}" \
  --build-path "${build_path}" \
  -Xswiftc "-sdk" -Xswiftc "${sdk_path}" \
  -Xswiftc "-target" -Xswiftc "${target_triple}" \
  -Xcc "-target" -Xcc "${target_triple}" 

Notice that the -target flags are specified for the Swift compiler and the clang compiler to ensure that the correct targets are built. Without these flags, we cannot even compile. (I tried adding it for the C++ compiler, but that did not change the behavior.)

Generate documentation.

  • Create a doc directory.
  • Implement a process for generating rule and library documentation from the Skylark code.

Enable select integration tests once `libswift_Concurrency.dylib` loading error in Swift is addressed

Description

The local_package and vapor examples fail when bazel test //... is executed. It appears to be related to changes made in a recent Swift release introducing the new concurrency model. For now, the related integration tests have been disabled.

Error Message

Library not loaded: /usr/lib/swift/libswift_Concurrency.dylib
  Referenced from: /var/tmp/_bazel_chuck/1ed0234260fd6dc70b7f4778a55f4296/sandbox/darwin-sandbox/306/execroot/vapor_example/bazel-out/darwin-fastbuild/bin/Tests/AppTests/AppTests.test-runner.sh.runfiles/vapor_example/Tests/AppTests/AppTests.xctest/Contents/MacOS/AppTests.xctest/Contents/MacOS/AppTests
  Reason: image not found

Related Links

Repo name must match product

Consider the following:

spm_repositories(
    name = "spm",
    // ... other attrs ...
    dependencies = [
        spm_pkg(
            "https://github.com/apple/swift-protobuf.git",
            exact_version = "1.19.0",
            products = ["SwiftProtobuf"],
        ),
    ],
)

Of course that's pointing to SwiftProtobuf's Package.swift, and it uses the SwiftProtobuf product from that configuration. However, when running a build with SPM, I get:

Repository rule spm_repositories defined at:
  /private/var/tmp/_bazel_sam/df355283ea71b0f4452e4b8a830191c8/external/cgrindel_rules_spm/spm/private/spm_repositories.bzl:693:35: in <toplevel>
ERROR: An error occurred during the fetch of repository 'spm':
   Traceback (most recent call last):
	File "/private/var/tmp/_bazel_sam/df355283ea71b0f4452e4b8a830191c8/external/cgrindel_rules_spm/spm/private/spm_repositories.bzl", line 691, column 30, in _spm_repositories_impl
		_configure_spm_repository(repository_ctx, pkgs, env)
	File "/private/var/tmp/_bazel_sam/df355283ea71b0f4452e4b8a830191c8/external/cgrindel_rules_spm/spm/private/spm_repositories.bzl", line 629, column 49, in _configure_spm_repository
		product = pds.get_product(pkg_descs_dict[pkg_name], product_name)
Error: key "swift-protobuf" not found in dictionary
ERROR: /Users/sam/Workspace/sample/WORKSPACE:929:17: fetching spm_repositories rule //external:spm: Traceback (most recent call last):
	File "/private/var/tmp/_bazel_sam/df355283ea71b0f4452e4b8a830191c8/external/cgrindel_rules_spm/spm/private/spm_repositories.bzl", line 691, column 30, in _spm_repositories_impl
		_configure_spm_repository(repository_ctx, pkgs, env)
	File "/private/var/tmp/_bazel_sam/df355283ea71b0f4452e4b8a830191c8/external/cgrindel_rules_spm/spm/private/spm_repositories.bzl", line 629, column 49, in _configure_spm_repository
		product = pds.get_product(pkg_descs_dict[pkg_name], product_name)
Error: key "swift-protobuf" not found in dictionary

Notable from that:
Error: key "swift-protobuf" not found in dictionary

Forking, and then renaming the repo, to SwiftProtobuf, fixed this particular error for me

Unable to build certain packages

Hey there rules_spm authors,

Firstly, love this project -- iOS dependencies are tough in Bazel land and this makes it better. Thank you for your hard work on this rule set. I have been using simple Swift packages like Logging with no issues, but adding more complex Swift packages like this one seem to produce errors:

ERROR: An error occurred during the fetch of repository 'swift_pkgs':
   Traceback (most recent call last):
	File "/private/var/tmp/_bazel_sam/df355283ea71b0f4452e4b8a830191c8/external/cgrindel_rules_spm/spm/private/spm_repositories.bzl", line 708, column 30, in _spm_repositories_impl
		_configure_spm_repository(repository_ctx, pkgs, env)
	File "/private/var/tmp/_bazel_sam/df355283ea71b0f4452e4b8a830191c8/external/cgrindel_rules_spm/spm/private/spm_repositories.bzl", line 579, column 52, in _configure_spm_repository
		resolve_out = repository_utils.exec_spm_command(
	File "/private/var/tmp/_bazel_sam/df355283ea71b0f4452e4b8a830191c8/external/cgrindel_rules_spm/spm/private/repository_utils.bzl", line 50, column 13, in _execute_spm_command
		fail(err_msg_tpl.format(
Error in fail: Resolution of SPM packages for swift_pkgs failed. args: ["xcrun", "swift", "package", "resolve", "--build-path", "spm_build"]
error: the manifest is missing a Swift tools version specification; consider prepending to the manifest '// swift-tools-version:5.5.0' to specify the current Swift toolchain version as the lowest Swift version supported by the project; if such a specification already exists, consider moving it to the top of the manifest, or prepending it with '//' to help Swift Package Manager find it

This error surfaces whether I depend on the Swift package or not; seemingly adding it to the WORKSPACE is enough to trigger it. I am happy to provide a repro or sample Swift packages where this occurs

Update the Linux toolchain so that the target architecture, OS, vendor and abi are not hardcoded

Today, the Linux toolchain is hardcoded to produce x86_64-unknown-linux-gnu. This mimcs what rules_swift does. Ideally, this would be evaluated using Bazel's toolchain mechanism (#50).

However, until this can happen, we could at least mirror the current target triple from the host. This can be retrieved by running swift --version.

$ swift --version
Swift version 5.4.2 (swift-5.4.2-RELEASE)
Target: x86_64-unknown-linux-gnu

However, given that rules_swift does not support other platforms, implementing it in rules_spm did not seem worth doing at this point.

unknown option --build-path

Hi ๐Ÿ‘‹

When using the rules I am getting the following error:

Error in fail: Resolution of SPM packages for swift_pkgs failed.
error: unknown option --build-path; use --help to list available options

Issue seems to be when running swift package resolve --build-path here:

["swift", "package", "resolve", "--build-path", spm_common.build_dirname],

Are there any minimum swift version requirements for the rules? Currently in my machine:

swift -version
Apple Swift version 5.3.2 (swiftlang-1200.0.45 clang-1200.0.32.28)

Thanks! ๐Ÿ™

Add example that uses external SPM that has its own external deps (e.g. vapor/vapor)

Tasks

  • Create modulemap parser written in Starlark, no submodule support (Starlark does not support recursive calls).
  • In spm_repository, use the modulemap parser to read modulemaps provided in source to find the public header files for a module. If no modulemap, do our default behavior (e.g. look for include directory and assume all of those are public headers).
  • Write spm_clang_module entries with the header and source files specified.
  • Fix unable to clone error in SPM build.
  • Apply the header copy and module map write for deps of dependent package.
  • Update the Github workflow to include the vapor example.

Document best practices for declaring spm_repositories

My recommendation (I need to make sure that this is documented) is to declare all of the packages that have library modules, which will be used as dependencies in your project, in a single spm_repositories. Any packages with binary/executable products that you want to use can be specified in separate spm_repositories.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.