peti/hydra-tutorial

Sample configuration files for setting up your own Hydra continuous integration server.

How to set up your own Hydra Server

For those who enjoy watching technical screencasts, there's also a video about this subject available at https://www.youtube.com/watch?v=RXV0Y5Bn-QQ.

This repository contains a complex'ish example configuration for the Nix-based continuous build system Hydra that new users can use to get started. The file hydra-common.nix defines basic properties of a VBox-based virtual machine running NixOS 16.03, which hydra-master.nix extends to configure a running Hydra server. hydra-slave.nix, on the other hand, configures a simple build slave for the main server to delegate build jobs to. Finally, hydra-network.nix ties those modules together into a network definition for Nixops.

To run these examples quickly with nixops on your local machine, you'll need

• hardware virtualization support,
• 8+ GB of memory,
• NixOS and Nixops installed.

Also, your configuration.nix file should include:

  virtualisation.virtualbox.host.enable = true;

If those pre-conditions are met, follow these steps:

1. Generate an SSH key used by the Hydra master server to authenticate itself to the build slaves:

   $ ssh-keygen -C "[email protected]" -N "" -f id_buildfarm

2. Set up your shell environment to use the nixos-16.03 release for all further commands:

   $ NIX_PATH="nixpkgs=https://github.com/nixos/nixpkgs-channels/archive/nixos-16.03.tar.gz"
   $ export NIX_PATH

3. Start the server:

   $ nixops create -d hydra hydra-network.nix
   $ nixops deploy -d hydra

4. Ensure that the main server knows the binary cache for nixos-16.03:

   nixops ssh hydra -- nix-channel --update

If all these steps completed without errors, then nixops info -d hydra will tell you the IP address of the new machine(s). For example, let's say that the hydra machine got assigned the address 192.168.56.101. Then go to http://192.168.56.101:8080/ to access the web front-end and sign in with the username "alice" and password "foobar".

Now you are ready to create projects and jobsets the repository contains the following examples that you can use:

• jobset-nixpkgs.nix
• jobset-libfastcgi.nix
• jobset-jailbreak-cabal.nix

The last jobset performs several Haskell builds that may be quite expensive, so it's probably wise not to run that on virtual hardware but only on a real sever.

Miscellaneous topics

• How to disable binary substitutions for higher evaluation performance.

• How to run emergency garbage collections:

  $ systemctl start hydra-update-gc-roots.service
  $ systemctl start nix-gc.service

• "Shares" are interpreted as follows: each jobset has a "fraction", which is its number of shares divided by the total number of shares. The queue runner records how much time a jobset has used over the last day as a fraction of the total time and then jobsets are ordered by their allocated fraction divided by the fraction of time used i.e. jobsets that have used less of their allotment are prioritized.

hydra-tutorial's Issues

Setting KnownHosts for SSH

Hydra queue runner doesn't automatically accept new hosts (due to default behavior in OpenSSH which is to ask for accepting the new host).

There are two ways to do that automatically:

1. by setting sshPublicHostKey which should be base64 encoded fingerprint line (not documented, but see commit NixOS/hydra@2a7fbd5), I will push a commit to enable setting this in NixOS
2. by setting programs.ssh.extraConfig = "StrictHostKeyChecking no"; in hydra-master

I chose the latter because it's a very low security tradeoff. SSH will still bail out if there is host mismatch, but it will automatically accept new hosts.

If hosts/key is hacked on first connection I think you're in trouble anyway.

Use store_uri

Last few months Hydra uses store_uri like here https://github.com/input-output-hk/pos-prototype-deployment/blob/master/modules/hydra-master.nix#L48 to specify store settings. The tutorial should follow that once Hydra is upgraded in nixpkgs.

The option `services.hydra' defined in '.../hydra-network.nix' does not exist

I followed README.md and got to this step (but no further):

$ nixops deploy -d hydra
building all machine configurations...
error: The option `services.hydra' defined in `/home/bfo/forks/hydra-tutorial/hydra-network.nix' does not exist.
(use ‘--show-trace’ to show detailed location information)
error: unable to build all machine configurations

It's due to NixOS/hydra@633724c ("services.hydra -> services.hydra-dev").

Lacks a signature

I've followed the tutorial to setup a Hydra cluster. Thanks again!

There's a peculiar error going on that I don't understand how exactly it should be addressed.

Aborted: imported archive of ‘/nix/store/nj34rl1sn86b50l940qgnmrcr6wbyal2-snabb-manual’ lacks a signature at https://hydra.snabb.co/build/16

Here's my Hydra master file: https://github.com/snabblab/snabblab-nixos/blob/master/modules/hydra-master.nix
And slave: https://github.com/snabblab/snabblab-nixos/blob/master/modules/hydra-slave.nix

I noticed Mayflower also generates another key, not sure if this is relevant: https://github.com/mayflower/mf-nixops/blob/master/component/hydra.nix#L65

cc @peti

Hydra build may fail because of the state of the current hydra master

For those trying this out and getting a build failure, this was the latest revision working for me today (2016-03-28):

let

  hydraSrc = builtins.fetchTarball https://github.com/nixos/hydra/archive/993647d1e3b43f1f9b7dc2ebce889b475d156bb9.tar.gz;

Improve password handling

Instructions how to set-up passwords in a secure manner have been posted at http://permalink.gmane.org/gmane.linux.distributions.nixos/20368.