svanderburg/disnix

Disnix: A Nix-based distributed service deployment tool

disnix's Issues

#nixos is a poor IRC channel to use for support

Well... basically as it says? The #nixos channel moves far too quickly; any question gets lost long before anyone who knows anything about Disnix might be able to help. #disnix is free, so maybe use that.

===

Also, my plea for help:

I'm attempting, as a simple test-case, to setup Mediawiki via Disnix. It hasn't gone too well so far; the manual doesn't make it clear what should go in which files, some files are referenced but never defined, and so forth. I nevertheless was able to write something that looks right, but...

See https://brage.info/disnix/

When I run disnix-manifest -s services/services.nix -i production/infrastructure.nix -d production/distribution.nix (or disnix-env), I get the following error:

error: attribute 'outPath' missing, at /nix/store/qalpkiap82fq9r8l60rhwd59r76kyr6j-disnix-0.8/share/disnix/lib.nix:168:39
(use '--show-trace' to show detailed location information)

I have no idea what I'm doing wrong.

Typo in Disnix User's Guide "11.2. Deploying a system locally"

In Disnix User's Guide "11.2. Deploying a system locally" section,

$ export DISNIX_CLIENT_INTERFACE=disnix-runactivity

should be

$ export DISNIX_CLIENT_INTERFACE=disnix-run-activity

Defining services is punishingly complicated

Not sure whether to file that against disnix or dysnomia. Also I should warn in advance that I'm only one week into the Nix world and clearly not Nix-minded enough yet.

If one wants to use Disnix to run a process that requires even a single command line argument, e.g. iperf -s, one must jump through so many hoops it's not even funny.

If I understood correctly, launching it without the -s would've required just iperf = { name = "iperf"; pkg = pkgs.iperf; type = "process"; };. But in order to add that little -s one must:
0. Specify type = "wrapper";.

1. Write a wrapper for it (your way seems to be: create a directory, write a wrapper, reuse a templating Makefile, compile (!) it)
2. Override a derivation and build it with the wrapper baked in (I can't even grasp how at the moment).

Am I wrong? Can the process be streamlined somehow? At the moment it's easier to overrideAttrs on the package and patch the sources to imply that '-s' than to make a wrapper.

Ideally I'd like to specify type = "process"; args = ["-s"]; for this trivial operation and something like

baseDir = "/var/lib/service";
type = modularWrapper {
  collectGarbage = ''
    rm -r $baseDir
  '';
  systemdUnit = ''
    [Unit]
    After=disnix.service

    [Install]
    WantedBy=dysnomia.target

    [Service]
    Environment=PATH=${dysnomia.systemdPath}
    Type=simple;
  '';
};

for more complex ones. Is this possible?

Disnix manual not found

404 Not Found
File /nix/store/jxq2xklvry5nlgzaf0swhrydcchviq80-disnix-tarball-0.7/share/doc/disnix/manual has disappeared.
Hydra 0.1.1234.abcdef (using nix-1.12pre5413_b4b1f452

https://hydra.nixos.org/build/54580779/download/2/manual

A disnix manual example appears to be missing crucial parts

"Example 4.6. Simplified intra-dependency composition for the StaffTracker" from section "4.3.3. Intra-dependency composition" appears to be missing some code and is not a well-formed nix expression on its own.

The example in question with foot notes removed:

{system, pkgs}:

let
  callPackage = pkgs.lib.callPackageWith (pkgs // self);
  
  self = {
  ### Databases
    rooms = callPackage ../pkgs/databases/rooms { };
  
    staff = callPackage ../pkgs/databases/staff { };
  
    zipcodes = callPackage ../pkgs/databases/zipcodes { };

  ### Web services + Clients
    ZipcodeService = callPackage ../pkgs/webservices/ZipcodeService { };
  
    ZipcodeServiceClient = callPackage ../pkgs/webservices/ZipcodeServiceClient { };
    
    ...
  
  ### Web applications
    StaffTracker = callPackage ../pkgs/webapplications/StaffTracker { };
  };
}

The let expression is opened but not paired with an in and does not have a nix expression to modify the context of.
The trailing closing curly brace has no matching opening pair.

Something got lost in translation somewhere or wasn't finished up as intended?

Clarify relation to NixOps

It seems like a pretty FAQ-ish thing, and I'd put it in the readme or the docbook.

Note: clarify naming requirement for type=process

Note the mismatch between the name of the job and the variable it is referred to as "job" vs "jobXX".

# services.nix
  job = {
    name = "jobXX";
    pkg = pkgs.callPackage ./job-package.nix {};
    dependsOn = {};
    type = "process";
  };
#dist.nix
job = [ infrastructure.machine1];

This will fail with a difficult to understand error:

disnix-env -s services.nix -i infra.nix -d dist.nix
[coordinator]: Building manifest...
error: value is a string while a set was expected
(use '--show-trace' to show detailed location information)

Changing it to name="job" removes the error and makes the system work as expected. This is documented in the manual:

Every service has a canonical name, so that it is known to which one is referred from the inter-dependency arguments. This name must match the attribute name.

An error message pointing people to this possibility or detecting it (or removing this constraint?) would be helpful.

disnix-ssh-client import-snapshots doesn't use SSH_USER when copying to the remote tempdir.

See scripts/disnix-ssh-client.in#L481.

I haven't actually encountered or tried to encounter this issue.

Disnix Docbook link in README is dead

Disnix has a nice Docbook manual that can be compiled yourself. However, it is also available online.

The link is dead. 😞

shellcheck identifies 3 errors in disnix-ssh-client.in

Shellcheck does static analysis to produce style, info, warning, and error findings on shell scripts, and provides generally good explanations of why a particular finding could be or is an error.

Shellcheck's online version identified 3 errors, several warnings, and many informational findings in scripts/disnix-ssh-client.in.
It would be good to fix the errors, and review the warnings/info findings since shellcheck's static analysis isn't necessarily always going to be good enough to figure out when a warning/info finding could actually result in an error at runtime.

At a glance, all 3 errors look like they would be relevant to supporting whitespace in filenames & would otherwise go unnoticed.

How to cleanly delete a deployment?

Is there a command that sends a deactivate message to all services and deletes the coordinator profile?

I've been playing around with Disnixos and NixOps. nixops destroy and nixops delete remove the NixOps deployment, but the services that I deployed with Disnixos do not run their deactivation scripts, and I have to manually delete the Disnix manifest to get back to a completely clean slate.

Failure when $TMPDIR does not exist on remote

With a local $TMPDIR set (by nix-shell) to /run/user/1000 and no corresponding directory on the (Debian) remote, disnix-env fails:

mktemp: failed to create file via template ‘/run/user/1000/tmp.XXXXXXXXXX’: No such file or directory
Cannot transfer intra-dependency closure!

Worked around by setting TMPDIR=/tmp when invoking disnix-env.

Using Disnix 0.5.

disnix-deploy description in user manual appendix is for a different command

Under A.2. Utilities, disnix-deploy's DESCRIPTION section describes disnix-migrate instead of disnix-deploy.

Noticed this while I was reading the online manual linked in README.md.

Disnix does not activate services in dependsOn order

Disnix currently does not activate services in the order established by dependsOn, but it should. For instance, if a service a dependsOn b, b should be activated before a.

Due to this bug, I cannot deploy anything on my Disnix cluster, as activation of services fails due to them being activated in the wrong order.

Difference between "manifest" and "distributed derivation"

I'm looking at the 2 commands:

disnix-instantiate

disnix-manifest

What's the difference between the output of these 2 commands.

Can't get disnix to work on Ubuntu 14.04 LTS

So I installed things on a Ubuntu 14.04 LTS VM:

sudo apt-get update -y
sudo apt-get upgrade -y
sudo apt-get install htop curl -y

curl https://nixos.org/nix/install | sh
. /home/clouduser/.nix-profile/etc/profile.d/nix.sh

nix-env -i $(nix-build -E 'with import <nixpkgs> {}; dysnomia.override { jobTemplate = "direct"; }')
nix-env -f '<nixpkgs>' -iA disnix

sudo cp /nix/var/nix/profiles/default/etc/dbus-1/system.d/disnix.conf /etc/dbus-1/system.d/
sudo cp /nix/var/nix/profiles/default/share/doc/disnix/disnix-service.initd /etc/init.d/

sudo mv /etc/init.d/disnix-service.initd /etc/init.d/disnix
sudo chmod +x /etc/init.d/disnix

osrel() {
    cat /etc/os-release | grep -E "$1"
}

if [[ $(osrel "^NAME=" | grep Debian) ]]; then
    sudo systemctl daemon-reload
elsif [[ $(osrel "^NAME=" | grep Ubuntu) ]]; then
    if [[ $(osrel "^VERSION_ID" | cut -d "=" -f 2) =~ '"14.04"' ]];
    then
        sudo initctl reload-configuration
    else
        echo "NOT SUPPORTED: $(osrel "^VERSION_ID")"
        exit 1
    fi
fi


for n in 2 3 4 5; do
    sudo ln -s /etc/init.d/disnix-service.initd /etc/rc${n}.d/S06disnix-service;
done

sudo vim /etc/dbus-1/system.d/disnix.conf
sudo vim /etc/init.d/disnix

sudo chmod a+w /var/run
sudo /etc/init.d/disnix start
sudo /etc/init.d/disnix status

echo "DONE"

And it seems that the call to sudo /etc/init.d/disnix start somehow failed (status returns nothing and gives exit code 3).

@svanderburg thinks that the dbus name gets lost somewhere.

With this patch applied (git diff):

diff --git a/src/dbus-service/disnix-service.c b/src/dbus-service/disnix-service.c
index 288d7b9..c279524 100644
--- a/src/dbus-service/disnix-service.c
+++ b/src/dbus-service/disnix-service.c
@@ -84,6 +84,12 @@ static void on_name_acquired(GDBusConnection *connection, const gchar *name, gpo
 static void on_name_lost(GDBusConnection *connection, const gchar *name, gpointer user_data)
 {
     g_printerr("Name lost: %s\n", name);
+
+       if (connection == NULL)
+               g_printerr("No D-Bus connection has been established!\n");
+       else
+               g_printerr("A D-Bus connection seems to have been established!\n");
+
     exit(1);
 }

I get the following output:

$ ./result/bin/disnix-service 
The Disnix service is running!
Name lost: org.nixos.disnix.Disnix
A D-Bus connection seems to have been established

So at least something seems to work.

I'm not sure where the actual error is located, though.

links to the manual on hydra are broken

on both nixos.org/disnix and in README.md, the link to the disnix manual is "https://hydra.nixos.org/job/disnix/disnix-trunk/tarball/latest/download-by-type/doc/manual". This page says "File /nix/store/a2ihxrk5k7yr9pxxp93d0z20646xnb4q-disnix-tarball-0.7/share/doc/disnix/manual does not exist.", which is a bit disappointing.

Launching services

I've read through the documentation, but there's a part I don't understand.

When using NixOS by itself, in order to have services run and be enabled, I usually have to specify it in configuration.nix which deterministically sets up the service file for systemd and compiles/installs the package.

When using disnix, and specifying for a service to run, is it modifying the configuration.nix and then doing a nixos-rebuild, or is it bypassing the configuration.nix. If it is bypassing it, how does it setup new services without making NixOS do a rebuild (especially if all the config directories like /etc are read-only). If it isn't bypassing it, why bother with disnix, we could just use DisnixOS which would do the same thing (changing the configuration.nix and running system specific services).

Basically why have dysnomia or nixos when you have DisnixOS? NixOS service configuration already have service files just by specifying them inside the configuration.nix. Then one only needs to configure it from DisnixOS, and you'll have services running and activated in the NixOS instances!