add Nix and Darwin CI

[b-lang-org]

add Nix and Darwin CI

[thoughtpolice]

Good news, I've already got it working. My build on my WIP branch uses GitHub Actions + Nix to do all the dirty work: thoughtpolice@6bbf58d and thoughtpolice@3229b94 do the business, and can reliably and repeatably CI every commit in ~20 minutes. It should also be possible to extend it to work with macOS. Github has a 6 hour build limit for jobs, which is fantastic, so I expect it will be able to handle the forthcoming test suite.

It's of course up to @quark17 whether she wants to adopt this infrastructure; I feel like us Nix users have sort of swarmed into the repository, so I don't want to feel like we're just throwing a bunch of Nix code at her that will be impossible for her to maintain. Continuous integration is very important, so it's not something that should be taken lightly. That said, I offer this counter argument: GA + Nix is actually pretty darn good! Builds in CI and builds on a developer machine are identical, we can do caching reliably, we support all the needed dependencies, it works on any Linux distro, etc etc. The compiler should build reliably and repeatably which is the most important quality, and there are many opportunities to speed up things using Nix in the future. We could even use it to build and release re-usable packages for end users (much like the commercial bluespec binary tarballs) in the long run, if we get a little clever. I feel we should be able to automate away the entire release process, for instance.

(The commits I have above however are basically "fire-and-forget". And I plan on continuing to do more active contributing and cleanup, so it's not like I won't be around to help if something goes wrong with them!)

That said, it can't do everything. For example, it seems entirely reasonable that people will want Bluespec to work on Windows at some point, and Nix can't handle that yet. Then again, "Windows is a special case" isn't exactly new in the FOSS world, so having a special build target is probably OK...

[flokli]

Since #56, we have a GitHub Action building bsc on the two supported Ubuntu versions. I didn't yet look into Mac OS Builds.

@thoughtpolice I'm not sure about the WIP branch. I feel like it adds quite some complexity. Can we trim it down to be as small as the derivation in nixpkgs is now? Or even reuse it, and override src and patches? I assume we want to fix the STP build, or make it optional, to test the actual source, and not a fork of it…

[thoughtpolice]

While it may appear otherwise, the goal of the build infrastructure I wrote is NOT to strictly be a CI system, it's also to provide a working environment for people to work on Bluespec while using Nix. I don't think anything it adds is at all complex; it largely just adds some tooling to manage Nixpkgs snapshot versions, but that's pretty much it. All of this code is available in my bootstrap.nix project, which is about ~100 lines.

Sharing the same or similar code to the upstream package is nice in theory, but in practice, IME anyway, the needs of developer workflows are largely different than upstream ones, and keeping a reasonable level of customization while sharing the same "base" code is normally over-engineering. It works at the exact moment everything is perfectly synchronized, but becomes extremely annoying the second you have to customize nearly anything significant (see below). While it does require some level of duplication between upstream and the repository itself, this is, in my experience, hardly a big deal, and almost none of the difficulty in maintenance comes from it (it all comes from things like vendoring.)

Can we trim it down to be as small as the derivation in nixpkgs is now

It actually is the same as the one you wrote for Nixpkgs, more or less. All the actual extra code comes from managing the version of nixpkgs, the derivation itself is very similar.

Or even reuse it, and override src and patches?

No, and this is a bad idea anyway, and I would strongly suggest against any patch adding it. It makes it very annoying to actually change anything while working on the compiler itself if you use Nix, because the package description is located somewhere else -- for example, if I introduce a tool or some other dependency, unless I'm lucky enough to have an argument, I now have to go and patch Nixpkgs to either pass in a new argument to the bluespec derivation (buildInputs) that I can then use in a function call from this repository, or I have to just go ahead and completely override the derivation outright, which puts me back at square 1. In both cases I might have to have a modified Nixpkgs, so I need a custom NIX_PATH and patches on top of it.

But even if you thought it was a good idea, it also does not actually meaningfully replace what bootstrap.nix does. bootstrap.nix is also about Nixpkgs snapshot management; without that you just have to rely on whatever version of Nixpkgs is in NIX_PATH, and that's frankly just not a robust strategy at all.

[thoughtpolice]

Also: I think if we have simple Ubuntu CI like now, then Nix CI should purely be Linux only and just use one nix version/package channel, it's not as important. Since we already have Ubuntu in place, I think we should focus on Darwin next before Nix.

[bpfoley]

I'm really pleased to see the CI builds BTW. Thanks for getting them up and running. 😀

[bpfoley]

BTW I'd vote for something running the oldest GHC we'd want to consider supporting.

From a little sampling, stack supports GHC back to ghc-7.8.4; Ubuntu 14.04 has ghc 7.6.3, Ubuntu 12.04 has ghc 7.4.1, and Ubuntu 10.04 has ghc 6.12.1.

6.12.1 is probably going back a bit too far, but we do have #ifdefs in the codebase support GHC 6.08 and the like...

[flokli]

@bpfoley there's a bunch of GHC's we could test for in CI, see https://github.com/actions/setup-haskell - we could easily add this to the test matrix. On the other hand, it might make sense to about what versions it still makes sense to support - GHC 6.8.3 was released June 2008, almost 12 years ago…