palfrey/herder

Automagic scheduling of a convention

Herder

Herder is a tool for automagically generate schedules for your convention, given such information as "when", "who's coming" and "what events do they want to attend". It leverages OptaPlanner for constraint satisfaction solving to be able to figure out the best schedule it can given your constraints, or at least the one that violates the smallest number of them.

It's a Clojure(Script) Boot app, with a tiny little bit of Java for helping OptaPlanner along.

Getting started

1. Get Boot
2. boot dev
3. Goto http://localhost:3000

This will get you a running dev setup using h2.

Production build

dev boot target builds, tests and runs the system. There's also prod-build which makes a target directory with all the parts in and prod-run which assumes you've done prod-build and runs that.

Production setup

This uses Docker, PostgreSQL and Let's Encrypt, but if you set DATABASE_URL and HOST you can run this outside of Docker. See docker-compose.yml for information. We're using prebuilt images from Docker Hub using prod-build/prod-run

1. Find a suitable machine to run all of this on (I suggest DigitalOcean, because then I get referrer credit). Main tested environment is a Debian 8.3 x64 machine.
2. apt-get install vim curl
3. Follow https://docs.docker.com/engine/installation/linux/debian/#debian-jessie-80-64-bit
4. Check docker version and then follow https://github.com/docker/compose/releases instructions for latest release
5. git clone https://github.com/palfrey/herder.git
6. cd herder
7. Edit docker-compose-public.yml. Set VIRTUAL_HOST and LETSENCRYPT_HOST to your host (hereafter assumed to be "herder.example.com"). Set LETSENCRYPT_EMAIL to your email.
8. docker-compose -f docker-compose-public.yml up -d
9. Goto http://herder.example.com:3000, and make sure you've got the main "Conventions" page. If not, check docker logs -f herder_herder_1 for "Starting #'herder.systems/prod-system" at the end. This takes a bit to occur after the "Postgres is up" line
10. Goto https://herder.example.com. If that doesn't work, check docker logs -f letsencrypt-nginx-proxy-companion for no ERROR lines
11. If this all works, replace "ports:" section of "herder" in "docker-compose-public.yml" with an "expose:" section

  ports:
    - "3000:3000"

becomes

  expose:
  - "3000"

and docker-compose -f docker-compose-public.yml up -d, which will stop the port 3000 version being exposed to the outside world.

Backup

If you've done the Production setup, backing up your data is good. Here's an example way to backup the Postgres data

1. apt-get install postgresql-client ssmtp uudeview
2. Create backup.sh as follows (replacing the "From" and "To" addresses)

#!/bin/bash
PGPASSWORD=mysecretpassword pg_dump postgres --host localhost --username=postgres | gzip -c - > backup.sql.gz
echo -e "From: [email protected]\nTo: [email protected]\nSubject: Herder backup\n\n"|uuenview -a -bo backup.sql.gz |sendmail -t

and doing a chmod +x backup.sh 3. To then backup every night at 2am, crontab -e and add the following 0 2 * * * /root/herder/backup.sh (assuming that's the path to backup.sh)

herder's Issues

Add link to herder github

Support for multiple slots in a day for one event

Dockerised deploy

When multiple changes happen to a schedule, throw away all but the last

We've only got one schedule for a convention, so rather than recalculating it several times over, do it once. This will need rewriting of the current agent-based system, as it just tries to feed new entries into the todo and instantly do them...

Non-list schedule layout

Events with N of the number of players

There's an event, it supports X players, but it has N players, where N <= MX and N > (M-1)X. Make M instances of the event, splitting players appropriately. Provide info on which player goes where (and what the flexibility is)

Multiple columns for event person list

Renaming event

Renaming person

Renaming and editing dates for con

Fold conventions list into SPA

Everything else is a single page app, other than the first page

Error messages for things going wrong

The API returns errors nicely, but the front-end never displays them.

Multiple columns for person list

Availability info for people

Unbreak prod-system

Should use PostgreSQL via DATABASE_URL

Per player "what you're in" list/schedule

Preferred day support

Only really needed for the Pub, as things like "do the LARP on a day people are around" is dealt with by people's availability.

Events with multiple slots in a day need better preferred slot support

"Preferred slot" in these cases should refer to the first one, not later ones.

Don't have already added persons in "add person" list

Multiple slots in a day events should be sequential

They're not always

Asynchronous solver calls

API calls that cause the solver to run right now wait for it to finish, which can cause front end issues.

Indicate out-of-date schedule

Schedule page doesn't differentiate between out-of-date and up-to-date, which given the delay for it to update is annoying.