acowley/cabbage

A tool for caching cabal builds in a Nix store

cabbage's Issues

Symlink bin directories into the sandbox

For example, make sure it works with GLUtil.

local system libs

Sometimes you want to depend on a system library that is not yet on nixpkgs. That would require writing a .nix file for it, but I am not sure how to pass that lib to cabbage after having done that.

Fix frameworks on pure-darwin

Rather than the current business of passing -framework flags directly to GHC in the generated default.nix files, we should add with pkgs.darwin.apple_sdk.frameworks; [$frameworks] to systemDeps.

Missing dependencies for adjunctions

When I try to build a toyproject I'm getting the following error on nix-shell --command 'sh $setup'. I run cabbage -a before. I also tried deleting .cabbages, cabal.config, cabal clean and cabal sandbox delete but sadly with no effect. Generating a shell.nix using cabal2nix --shell . > shell.nix and then running nix-shell --command 'cabal build' works just fine. Sadly I don't really know what I am doing and this bash script is scaring me so I'm a bit lost. I'd be happy to provide more information if needed.

building path(s) ‘/nix/store/y1v7r6176rqc3f024gn217rs4kiy0ndg-haskell-adjunctions-4.2’
Warning: The package list for 'hackage.haskell.org' does not exist. Run 'cabal
update' to download it.
Resolving dependencies...
Configuring adjunctions-4.2...
cabal: At least the following dependencies are missing:
comonad ==4.*,
contravariant ==1.*,
distributive >=0.4 && <1,
free ==4.*,
mtl >=2.0.1 && <2.3,
profunctors ==4.*,
semigroupoids ==4.*,
semigroups >=0.11 && <1,
tagged >=0.7 && <1,
void >=0.5.5.1 && <1
builder for ‘/nix/store/wiww14fmb7c8q328d6701kdjfs5b1jdd-haskell-adjunctions-4.2.drv’ failed with exit code 1
error: build of ‘/nix/store/wiww14fmb7c8q328d6701kdjfs5b1jdd-haskell-adjunctions-4.2.drv’ failed
/home/moritz/.nix-profile/bin/nix-shell: failed to build all dependencies

Not quite sure which files are relevant so here is a selection :)
cabal.config

constraints: ConfigFile ==1.1.4,
             HUnit ==1.2.5.2,
             MissingH ==1.3.0.1,
             PSQueue ==1.1,
             QuickCheck ==2.7.6,
             adjunctions ==4.2,
             ansi-wl-pprint ==0.6.7.1,
             asn1-encoding ==0.9.0,
             asn1-parse ==0.9.0,
             asn1-types ==0.3.0,
             base16-bytestring ==0.1.1.6,
             base64-bytestring ==1.0.0.1,
             byteable ==0.1.1,
             case-insensitive ==1.2.0.4,
             cereal ==0.4.1.1,
             cipher-aes ==0.2.10,
             cipher-des ==0.0.6,
             cipher-rc4 ==0.1.4,
             clock ==0.4.1.3,
             connection ==0.2.4,
             cookie ==0.4.1.4,
             crypto-cipher-types ==0.0.9,
             crypto-numbers ==0.2.7,
             crypto-pubkey ==0.2.8,
             crypto-pubkey-types ==0.4.3,
             crypto-random ==0.0.8,
             cryptohash ==0.11.6,
             doctest ==0.9.13,
             errors ==1.4.7,
             hourglass ==0.2.8,
             hslogger ==1.2.8,
             hspec ==2.1.4,
             hspec-core ==2.1.4,
             hspec-discover ==2.1.4,
             hspec-expectations ==0.6.1.1,
             http-client ==0.4.8.1,
             http-client-tls ==0.2.2,
             http-types ==0.8.6,
             iCalendar ==0.4.0.2,
             kan-extensions ==4.2.1,
             lens ==4.8,
             lens-aeson ==1.0.0.3,
             logict ==0.6.0.2,
             mime ==0.4.0.1,
             mime-types ==0.1.0.5,
             network ==2.6.0.2,
             network-uri ==2.6.0.1,
             optparse-applicative ==0.11.0.2,
             parallel ==3.2.0.6,
             pem ==0.2.2,
             publicsuffixlist ==0.1,
             quickcheck-io ==0.1.1,
             reflection ==1.5.1.2,
             regex-base ==0.93.2,
             regex-compat ==0.95.1,
             regex-posix ==0.95.2,
             regex-tdfa ==1.2.0,
             regex-tdfa-rc ==1.1.8.3,
             securemem ==0.1.7,
             setenv ==0.1.1.3,
             smallcheck ==1.1.1,
             socks ==0.5.4,
             streaming-commons ==0.1.10.0,
             tasty ==0.10.1,
             tasty-hspec ==1.1,
             tasty-quickcheck ==0.8.3.2,
             tasty-smallcheck ==0.8.0.1,
             tf-random ==0.5,
             tls ==1.2.16,
             unbounded-delays ==0.1.0.9,
             wreq ==0.3.0.1,
             x509 ==1.5.0.1,
             x509-store ==1.5.0,
             x509-system ==1.5.0,
             x509-validation ==1.5.1,
             zlib ==0.5.4.2

default.nix

{ stdenv, lib, haskellBuildTools, pkgs, callPackage }:
let cabalTmp = "cabal --config-file=./.cabal/config";
    frozenCabbages = rec {
      ConfigFile = callPackage .cabbages/ConfigFile-1.1.4 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      HUnit = callPackage .cabbages/HUnit-1.2.5.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      MissingH = callPackage .cabbages/MissingH-1.3.0.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      PSQueue = callPackage .cabbages/PSQueue-1.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      QuickCheck = callPackage .cabbages/QuickCheck-2.7.6 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      adjunctions = callPackage .cabbages/adjunctions-4.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      ansi-wl-pprint = callPackage .cabbages/ansi-wl-pprint-0.6.7.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      asn1-encoding = callPackage .cabbages/asn1-encoding-0.9.0 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      asn1-parse = callPackage .cabbages/asn1-parse-0.9.0 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      asn1-types = callPackage .cabbages/asn1-types-0.3.0 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      base16-bytestring = callPackage .cabbages/base16-bytestring-0.1.1.6 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      base64-bytestring = callPackage .cabbages/base64-bytestring-1.0.0.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      byteable = callPackage .cabbages/byteable-0.1.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      case-insensitive = callPackage .cabbages/case-insensitive-1.2.0.4 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      cereal = callPackage .cabbages/cereal-0.4.1.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      cipher-aes = callPackage .cabbages/cipher-aes-0.2.10 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      cipher-des = callPackage .cabbages/cipher-des-0.0.6 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      cipher-rc4 = callPackage .cabbages/cipher-rc4-0.1.4 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      clock = callPackage .cabbages/clock-0.4.1.3 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      connection = callPackage .cabbages/connection-0.2.4 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      cookie = callPackage .cabbages/cookie-0.4.1.4 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      crypto-cipher-types = callPackage .cabbages/crypto-cipher-types-0.0.9 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      crypto-numbers = callPackage .cabbages/crypto-numbers-0.2.7 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      crypto-pubkey = callPackage .cabbages/crypto-pubkey-0.2.8 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      crypto-pubkey-types = callPackage .cabbages/crypto-pubkey-types-0.4.3 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      crypto-random = callPackage .cabbages/crypto-random-0.0.8 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      cryptohash = callPackage .cabbages/cryptohash-0.11.6 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      doctest = callPackage .cabbages/doctest-0.9.13 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      errors = callPackage .cabbages/errors-1.4.7 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      hourglass = callPackage .cabbages/hourglass-0.2.8 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      hslogger = callPackage .cabbages/hslogger-1.2.8 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      hspec = callPackage .cabbages/hspec-2.1.4 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      hspec-core = callPackage .cabbages/hspec-core-2.1.4 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      hspec-discover = callPackage .cabbages/hspec-discover-2.1.4 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      hspec-expectations = callPackage .cabbages/hspec-expectations-0.6.1.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      http-client = callPackage .cabbages/http-client-0.4.8.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      http-client-tls = callPackage .cabbages/http-client-tls-0.2.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      http-types = callPackage .cabbages/http-types-0.8.6 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      iCalendar = callPackage .cabbages/iCalendar-0.4.0.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      kan-extensions = callPackage .cabbages/kan-extensions-4.2.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      lens = callPackage .cabbages/lens-4.8 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      lens-aeson = callPackage .cabbages/lens-aeson-1.0.0.3 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      logict = callPackage .cabbages/logict-0.6.0.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      mime = callPackage .cabbages/mime-0.4.0.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      mime-types = callPackage .cabbages/mime-types-0.1.0.5 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      network = callPackage .cabbages/network-2.6.0.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      network-uri = callPackage .cabbages/network-uri-2.6.0.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      optparse-applicative = callPackage .cabbages/optparse-applicative-0.11.0.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      parallel = callPackage .cabbages/parallel-3.2.0.6 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      pem = callPackage .cabbages/pem-0.2.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      publicsuffixlist = callPackage .cabbages/publicsuffixlist-0.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      quickcheck-io = callPackage .cabbages/quickcheck-io-0.1.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      reflection = callPackage .cabbages/reflection-1.5.1.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      regex-base = callPackage .cabbages/regex-base-0.93.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      regex-compat = callPackage .cabbages/regex-compat-0.95.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      regex-posix = callPackage .cabbages/regex-posix-0.95.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      regex-tdfa = callPackage .cabbages/regex-tdfa-1.2.0 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      regex-tdfa-rc = callPackage .cabbages/regex-tdfa-rc-1.1.8.3 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      securemem = callPackage .cabbages/securemem-0.1.7 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      setenv = callPackage .cabbages/setenv-0.1.1.3 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      smallcheck = callPackage .cabbages/smallcheck-1.1.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      socks = callPackage .cabbages/socks-0.5.4 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      streaming-commons = callPackage .cabbages/streaming-commons-0.1.10.0 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      tasty = callPackage .cabbages/tasty-0.10.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      tasty-hspec = callPackage .cabbages/tasty-hspec-1.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      tasty-quickcheck = callPackage .cabbages/tasty-quickcheck-0.8.3.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      tasty-smallcheck = callPackage .cabbages/tasty-smallcheck-0.8.0.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      tf-random = callPackage .cabbages/tf-random-0.5 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      tls = callPackage .cabbages/tls-1.2.16 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      unbounded-delays = callPackage .cabbages/unbounded-delays-0.1.0.9 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      wreq = callPackage .cabbages/wreq-0.3.0.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      x509 = callPackage .cabbages/x509-1.5.0.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      x509-store = callPackage .cabbages/x509-store-1.5.0 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      x509-system = callPackage .cabbages/x509-system-1.5.0 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      x509-validation = callPackage .cabbages/x509-validation-1.5.1 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
      zlib = callPackage .cabbages/zlib-0.5.4.2 {
        inherit frozenCabbages haskellBuildTools pkgs;
      };
    };
    mkCmd = pkg: let nm = lib.strings.removePrefix "haskell-" pkg.name;
                     p = pkg.outPath;
                     pkgPath = ".cabal-sandbox/x86_64-linux-ghc-7.8.4-packages.conf.d";
                 in ''ln -sFf ${p}/${pkgPath}/*.conf $out/${pkgPath}/
                    '';
    mkSetupCmd = pkg: let nm = lib.strings.removePrefix "haskell-" pkg.name;
                          p = pkg.outPath;
                      in "ln -sFf ${pkg.outPath}/.cabal-sandbox/x86_64-linux-ghc-7.8.4-packages.conf.d/*.conf /home/moritz/code/haskell/trccbot-hs/.cabal-sandbox/x86_64-linux-ghc-7.8.4-packages.conf.d/\n";
in
stdenv.mkDerivation rec {
  name = "haskell-trccbot-hs-0.1.0.0";
  src = ./dist/trccbot-hs-0.1.0.0.tar.gz;
  cabbageDeps = with frozenCabbages; [ ConfigFile HUnit MissingH PSQueue QuickCheck adjunctions ansi-wl-pprint asn1-encoding asn1-parse asn1-types base16-bytestring base64-bytestring byteable case-insensitive cereal cipher-aes cipher-des cipher-rc4 clock connection cookie crypto-cipher-types crypto-numbers crypto-pubkey crypto-pubkey-types crypto-random cryptohash doctest errors hourglass hslogger hspec hspec-core hspec-discover hspec-expectations http-client http-client-tls http-types iCalendar kan-extensions lens lens-aeson logict mime mime-types network network-uri optparse-applicative parallel pem publicsuffixlist quickcheck-io reflection regex-base regex-compat regex-posix regex-tdfa regex-tdfa-rc securemem setenv smallcheck socks streaming-commons tasty tasty-hspec tasty-quickcheck tasty-smallcheck tf-random tls unbounded-delays wreq x509 x509-store x509-system x509-validation zlib ];
  systemDeps = (with pkgs; [  ]) ++
               lib.lists.unique (lib.concatMap (lib.attrByPath ["systemDeps"] []) cabbageDeps);
  propagatedBuildInputs = systemDeps;
  buildInputs = [ stdenv.cc ] ++ haskellBuildTools ++ cabbageDeps ++ systemDeps;

  # Build the commands to merge package databases
  cmds = lib.strings.concatStrings (map mkCmd cabbageDeps);
  setupCmds = lib.strings.concatStrings (map mkSetupCmd cabbageDeps);
  setup = builtins.toFile "setup.sh" ''

    # Takes a GHC platform string, an array of add-source dependency
    # directories, and a string of old timestamps. Produces a new
    # timestamp string.
    updateTimeStamps() {
      local -a DEPS=("''${!2}")
      local CUR_TIME=$(date +%s)
      local i
      local STAMPED
      for ((i = 0; i < "''${#DEPS[@]}"; ++i)); do
        STAMPED[$i]="(\"''${DEPS[$i]}\",$CUR_TIME)"
      done
      local LIST=$(printf ",%s" "''${STAMPED[@]}")
      LIST=''${LIST:1}
      local NEWSTAMP="(\"$1\",[$LIST])"
      if echo "$3" | grep -q "$1"; then
        echo "$3" | sed "s:(\"$1\",[^]]*\]):$NEWSTAMP:"
      elif echo "$3" | grep -q "]\\$"; then
        echo "$3" | sed "s:\]\$:,$NEWSTAMP]:"
      else
        echo "[$NEWSTAMP]"
      fi
    }
    eval "$setupCmds"
    ${cabalTmp} sandbox hc-pkg recache
    SRCS=($(cabal sandbox list-sources | sed '1,/^$/ d' | sed '/^$/,$ d'))
    OLDTIMESTAMPS=$(cat .cabal-sandbox/add-source-timestamps)
    updateTimeStamps "x86_64-linux-ghc-7.8.4" SRCS[@] "$OLDTIMESTAMPS" > .cabal-sandbox/add-source-timestamps
  '';

  builder = builtins.toFile "builder.sh" ''
    source $stdenv/setup
    mkdir $out

    if [ -d "$src" ]; then
      cp -R "$src"/* .
      #*/
      if [ -f $src/cabal.config.bak ]; then
        cp "$src/cabal.config.bak" $out
      fi
    else
      tar xf "$src" --strip=1
    fi

    chmod -R u+w .
    ${cabalTmp} sandbox --sandbox=$out/.cabal-sandbox init -v0
    mkdir -p $out/.cabal-sandbox/lib/x86_64-linux-ghc-7.8.4
    eval "$cmds"
    ${cabalTmp} sandbox hc-pkg recache

    ${cabalTmp} --builddir=$out/dist --bindir=$out/bin --libdir=$out/.cabal-sandbox/lib --with-gcc=$CC configure $(echo $NIX_LDFLAGS | awk -e '{ for(i=1;i <= NF; i++) { if(match($(i), /^-L/)) printf("--extra-lib-dirs=%s ", substr($(i),3)); } }')
    echo "Building..."
    ${cabalTmp} --builddir=$out/dist build -v0
    ${cabalTmp} --builddir=$out/dist copy
    ${cabalTmp} --builddir=$out/dist register
    ${cabalTmp} --builddir=$out/dist clean || true
  '';    
  meta = {
    description = "";
  };
}

Adding build-depends from executable section

I'm trying to run the demos from acowley/Frames.

cd ~/sources/Frames
cabbage
nix-shell --pure -p haskellngPackages.cabal-install haskellngPackages.ghc
cabal repl
*Frames> :l demo/Main.hs

demo/Main.hs:6:8:
    Could not find module `Lens.Family'
    It is a member of the hidden package `lens-family-core-1.2.0'.
    Perhaps you need to add `lens-family-core' to the build-depends in your .cabal file.
    Use -v to see a list of the files searched for.

demo/Main.hs:7:18:
    Could not find module `ListT'
    It is a member of the hidden package `list-t-0.4.3'.
    Perhaps you need to add `list-t' to the build-depends in your .cabal file.
    Use -v to see a list of the files searched for.
Failed, modules loaded: none.

I don't think this is the main aim of cabbage, but I bet it would be a useful use case and something not hard to work around... but could be wrong.

build tools

I have a project that requires c2hs. I tried making a cabbage.config containing

systemDeps:
  c2hs: c2hs

but this results in a warning and a possibly unrelated error from find

WARNING: cabal get did not unpack c2hs
Should we continue? [Y/n] 
find: warning: you have specified the -depth option after a non-option argument -name, but options are not positional (-depth affects tests specified before it as well as those specified after it).  Please specify options before other arguments.

find: paths must precede expression: 1
Usage: find [-H] [-L] [-P] [-Olevel] [-D help|tree|search|stat|rates|opt|exec] [path...] [expression]
WARNING: Couldn't find build tool: c2hs

when running cabbage. When I try to run nix-shell --command 'sh $setup' nevertheless I get the following error

error: anonymous function at "/home/moritz/code/haskell/reactand/default.nix":1:1 called without required argument ‘c2hs’, at "/nix/store/lgjabak42v2hp51dvj3fc0c3pxpnkk9h-nixpkgs-15.06pre63073.f0311f9/nixpkgs/lib/customisation.nix":58:12

I have c2hs in my path but nix probably needs it's own version.