pier,judah

Support setting `ghc-options`

Either from the command-line and/or set in stack.yaml.

Stack documentation here:
https://docs.haskellstack.org/en/stable/yaml_configuration/#ghc-options

Build from stack.yaml file

Take configuration like packages and lts from a stack.yaml file

Store `_pier/archive` under $HOME

Make _pier/archive link to the $HOME directory, so different projects can share the cached files.

Consider "override" mechanism for Stackage packages

Some packages are set up with strange edge cases; for example, hslua doesn't list the include file lua.h in the includes clause, only in the extra-src-files. Rather than implementing special cases for each one, consider hard-coding a list of packages with explicit overrides.

User documentation

Add documentation of how to invoke stake, what the commands do (e.g. clean vs clean-all), and where output files are located.

Preprocess use stake-built executables

Use stake-created executables for happy, alex, hsc2hs, etc.

hsc2hs should be provided by the GHC install. alex/happy can be built as separate packages.

Allow lower-cased `main-is`

If an executable has a lower-cased main-is, stake will output an error:

Error when running Shake build system:
* Top-level action/want
ModuleName.fromString: invalid module name "main"
CallStack (from HasCallStack):
  error, called at libraries/Cabal/Cabal/Distribution/ModuleName.hs:81:44 in Cabal-1.24.2.0:Distribution.ModuleName

Example package: yaml

Support c2hs as a preprocessor

Add c2hs to the list of supported preprocessors.

Find a package using it, and confirm that it builds after this change.

Option for linking executables statically

It will make rebuilds take up significantly less space, and possibly take less time too. (We might be able to avoid the "dynamic-too" and "ar" invocations if we're only building shared libraries with "-shared".)

We'll want to add a linkArtifact analogue of copyArtifact, and link all dylibs in one location (similar to the workaround that Cabal/Nix/etc use for macOS Sierra).

Make dynamic linking the default
Allow static linking as another option
Work around the macOS Sierra linking issue

`cryptonite` with `lts-9.10` doesn't build

I can successfully run:
stack --resolver lts-9.10 install cryptonite

Also, this succeeds:

*Development.Stake> :main build -p lts-9.6 cryptonite
Downloading plan...
Downloading cryptonite-0.23
Downloading memory-0.14.8
Downloading foundation-0.0.15
Downloading basement-0.0.2
Building basement-0.0.2
Building foundation-0.0.15
Building memory-0.14.8
Building cryptonite-0.23
Build completed in 1:09m

However, this then failed:

*Development.Stake> :main build -p lts-9.10 cryptonite
Downloading plan...
Downloading foundation-0.0.16
Downloading basement-0.0.3
Building basement-0.0.3
Building foundation-0.0.16
Building memory-0.14.8
Building cryptonite-0.23

.stake/downloads/hackage/cryptonite-0.23/Crypto/Hash/Types.hs:56:22: error:
    • No instance for (ByteArrayAccess (F.UArray Word8))
        arising from the 'deriving' clause of a data type declaration
      Possible fix:
        use a standalone 'deriving instance' declaration,
          so you can specify the instance context yourself
      There are instances for similar types:
        instance basement-0.0.2:Basement.PrimType.PrimType ty =>
                 ByteArrayAccess (basement-0.0.2:Basement.UArray.Base.UArray ty)
          -- Defined in ‘[email protected]:Data.ByteArray.Types’
    • When deriving the instance for (ByteArrayAccess (Digest a))

Add a `stake exec` command

Add a command that lets you run within a preassembled environment, like stake exec or nix-shell.

Reorganize modules

reorganize and consolidate the individual modules. There's some overlap in their functionality.

Be more tolerant of running in subdirectories

If run in a subdirectory, find the stack.yaml file and build relative to it.

Also figure out sane behavior if the path is set explicitly by the command-line flag.

Make local "frozen" version of stackage/hackage

Enable working with a local version of a subset of hackage/stackage:

specific LTSes
specific packages
Possibly: GHC version (see #8)

Enable generating such a set from the closure of a small set of packages (e.g., what's tested in CI).

Useful for developing on airplanes, saving bandwidth costs, etc.

Support Alex

Support alex as a preprocessor.

Find a package that requires this, and confirm it builds after this change.

Link warning when building aeson

Building aeson-1.1.2.0
unused rules: 3
Warning: ignoring unrecognised input `.stake/build/lts-9.6/packages/aeson-1.1.2.0/o/.stake/downloads/hackage/aeson-1.1.2.0/cbits/unescape_string.dyn_o'

Use a `pier.yaml` file

Use its own file rather than stack.yaml, since they're going to diverge in which fields they support.

Use a docker image of stackage deps

Stackage has a handy dockerfile that we could use for CI and/or local testing:
https://github.com/fpco/stackage/blob/master/Dockerfile

Use hermetic `happy` and `alex`

happy and alex can't be used for codegen yet because their data-files are generated by a custom setup script.

More robust sandboxing

Either use a "true" sandbox (potentially hard), or just figure out how to do lndir more quickly (for example by caching the set of deps under a directory).

Link libraries

Implement `pier repl`

The command pier ghci ("repl"?) should build all dependencies of the given target(s) and then load the source files into ghci.

Don't use uuidgen

Don't require the separate program uuidgen. Either use an in-process implementation, or just generate a random number, or do something simpler.

Better unit testing

Add explicit unit tests, instead of (or in addition to) the mass-package-compile that's currently in the CircleCI scripts.

Ideally this would let us decrease the time for CircleCI to run significantly, for example by avoiding large packages like haskell-src-exts and language-c.

Build executables

Build and link executables for each package.

Consider not checksumming every input

Consider using Shake's ChangeModtimeAndDigest rather than just ChangeDigest. Reason through the effect (if any) on Command, and test it out.

Try using a Dockerfile in circleci

The majority of rebuilding is just in saving the cache. We should be able to do better.

Support `hpack` for local packages

Support using hpack to build local packages.

Generate the .cabal file in a separate Artifact, to avoid polluting the output directory.

Restrict which binaries the rules can call

shake will pull binaries from the current $PATH, which is more lenient than we might like.

One option is to wrap calls to Distribution.Shake.command in setEnv, but that seems a little heavy-weight (and possibly not thread-safe). Another option is to wrap the whole program in it.

Use a hermetic GHC

Download and use GHC from stack-setup-2.yaml.

Integrate with Nix

Consider a feature similar to stack --nix which uses Nix for external dependencies.

This appears to be necessary for running pier on NixOS.

Separate utilities into a separate library

Separate utilities like Command and Persistent into a separate library (or libraries), to help demonstrate their generality.

Haddock support

Allow stake to generate haddocks for a given set of libraries.

Maybe also just a "shallow" version that doesn't generate it for dependencies.

False-positive "Building {pkg}" notifications

After a make clean, stake outputs "Building {pkg}" even when it's not going to re-run any of the Commands.

On the one hand, this is useful for debugging. On the other, it's probably more helpful to report only when we're actually re-running GHC et al. (Similar to bazel's per-action progress message.)

Support Cabal-2

Currently we build against Cabal-1.24.0.2, since that's what's in lts-9.*. We should support newer Cabal (2.*.*.*) at some point.

Re-implement with Command

Also figure out a good hermetic story for rebuilds of GHC packages that don't recompile unnecessarily...

Silence GHC install

Hide stderr output from the GHC install, unless something fails.

Look for a way to do this more generically (without losing the ability to see ghc warnings)

Parse `ShakeOptions`

Parse options from the command-line (add optparse-applicative for it).

Any shake option (via something like --shake-arg=)
some specific ones like verbosity and num. threads

Optimization levels

Allow -O0, -O1, -O, -O2

(Right now it defaults to -O0.)

Build cpphs automatically and use it hermetically

See for example the Agda package which declares it in build-tools:

agda/agda#2567
https://github.com/agda/agda/blob/master/Agda.cabal#L106

Code audit and refactor/documentation

Review all the code and refactor/document it, now that the structure is more stable.

Implement `pier test`

Implement a pier test command that builds and runs individual tests.

Also add an option to build but not run them; maybe do that by default.

Decide whether the actual runs should be cached or not (or if that should be another option).

Build local packages

Allow specifying local packages and building them against an LTS.

Not sure if config should be stack.yaml or a Haskell script.

Figure out story for Custom setup scripts

How should we support packages like proto-lens or gl that generate Haskell modules with a custom Setup file? Allow custom user rules? Actually just run their Setup script?

Some specific packages:

happy, alex: generate template data-files by compiling and running a program with different CPP flags.
pandoc: installs man page
semigroupoids, comonad, distributive: use cabal-doctest
lens: uses cabal-doctest, and also installs an image to be used by the haddocks
entropy: runs a test C program, not sure for what purpose
darcs: generates a custom Version.hs module for embedding more specific version information; also custom logic for linking against libiconv, similar to what used to be in haskeline
gtk2hs: packages like glib use gtk2hs-buildtools:Gtk2HsSetup

judah / pier Goto Github PK

pier's People

Contributors

Stargazers

Watchers

Forkers

pier's Issues

Recommend Projects

Recommend Topics

Recommend Org