Update `rules_spm` to build fetched dependencies using Bazel instead of Swift Package Manager

[cgrindel]

Update `rules_spm` to build fetched dependencies using Bazel instead of Swift Package Manager

[cgrindel]

Looks like I will need to parse the Package.swift to get any C-specific settings/options.

Example: swift-nio with CSetting
Example: Yams with CSetting

Example code to parse Package.swift.

[cgrindel]

Notes on getting libwebp in interesting_deps example to build:

• Need to add the following includes to find the private headers.

    includes = [
        "include/webp",
        "libwebp",
        "libwebp/src",
    ],

[cgrindel]

Over the past week or so, I have been working on a proof-of-concept to understand what it would take to introduce a build_mode to spm_repositories. Setting this attribute to bazel would change how rules_spm works underneath the covers. Instead of building the dependent packages using SPM, it would generate Bazel build files allowing the targets to be built using the normal Bazel toolchains. I was able to successfully build and use Swift packages with Swift targets and generic clang targets. Unfortunately, configuring clang targets with custom build settings requires more information than a package description provides.

To be able to properly build clang targets with custom build settings, one needs to retrieve and apply these settings (e.g. defines, public header paths). The current scheme for downloading and reading these manifest files during the repository fetch phase requires code that can parse the manifest and present the information in a means that is readable by Starlark code. Unfortunately, there is no Swift package manifest parser written in Starlark. One can write a parser in Starlark and maintain it as SPM evolves. The question is whether that is the best path forward.

This conundrum is not new to this project. The same decision had to be made when figuring out how to read data from module.modulemap files. At the time, we opted to write a modulemap parser in Starlark thinking that the format was fairly mature and did not appear to be changing much.

To avoid having to write a custom parser, I wrote a simple Swift binary that reads a Swift package manifest and dumps information about it to JSON. The JSON is then read by Starlark code. The approach works. The problem is how one gets and uses the binary from code that is run during Bazel's fetch phase. As of this writing, the current code uses swift run to build and execute the parser. This works OK if there is only one spm_repositories declaration. However, if there are multiple declarations, one will experience an error due to two Swift processes trying to build the same package. I could work around this by ensuring that two processes never build the same code simultaneously. However, that would require duplicating the code and the compilation (ugh) or inventing some kind of semaphore that only allows one build to run (double ugh).

So, I am left with one of a few options:

1. Solve the duplicate build problem for spm_parser. (Yuck)
2. Write a Swift package manifest parser in Starlark. This would allow rules_spm to provide a repository rule that magically does the heavy lifting that it does today.
3. Go completely different direction with rules_spm. Instead of providing a repository rule, it would provide an executable target that generates Bazel build files similar to google/cargo-raze. (Thanks to @keith for making me aware of this project.)

[cgrindel]

Created #157 asking the community how they would like to move forward.