lionsphil/plumage

A web frontend for the Web Polygraph proxy testing tool

plumage's Issues

No real progress feedback while a run is executing

The feedback mechanisms for a run in progress are not yet implemented, beyond running/completed. There are plans for plumage_run to aggregate the logs and the Client to provide a server-sent events feed to the UI. This is very important when doing test configuration development where Polygraph may be throwing errors, since currently the only way to see them is to SSH onto the Client and dig them out of the state directory. There's also work to be done around this area in properly reporting runs which were aborted.

It should also be easier to see when a Client/Server pair are busy to avoid scheduling another run on top of them.

Internal error reporting should be better

At the moment, when a web service API call fails, Plumage generally just throws and ends up displaying a 500 error, with some detail in the logs. While semi-appropriate for a public-facing system, this is a technical internal one, and having "contacting the Master component failed", or "Master reports Client failed to spawn plumage_run" or such is more useful.

Installation documentation and metadata is poor

The existing ./deploy and example configurations are a bit rough to get set up in an environment. Could do with:

• Recommended starting point/philosophy document (hosts, network layout)
• Correct MakeMaker metadata about dependencies
• Debian/Ubuntu packaging metadata to make packages of each component (with dependencies)
  • Move away from running under the same user that owns the program
  • Move out of /opt
  • Configuration in or symlinked to /etc
• Change the default port for each component so you can run them all on one machine if you want
• A more complete INSTALL guide to use the above

Ideally it should be a doddle to make Puppet put (and update) Plumage on things.

Run delete/abort fails when Polygraph is in some broken states

Plumage does not recover well from some inconsistent states where runs have failed, because attempts to clean up across all the machines involved become fatal. For example, if there's a problem with the connectivity between the client and server, e.g. the proxy under test is misconfigured or misbehaving, the polygraph server side can time out and shut down.

Delete actions on the master should tolerate the client or server reporting that they are unable to delete the run in question and still continue trying to clean up the run locally.

In the meantime, the workaround is to (in order, since some steps may unblock later steps):

1. SSH onto the server, check for and kill any running polygraph processes, and clean out /run/plumage/server/.
2. Repeat for the client and /run/plumage/client/. If there were polygraph processes to kill, be patient; this may unjam the process and start reporting results back to the master.
3. SSH onto the master, look under /var/lib/plumage/configurations/N/runs/ (where N is in the URL of the configuration), and delete the directory matching the broken run number.

Redesign master/client/server role boundaries to allow multi-machine tests

Plumage's early design did not have distinct Master or UI roles, which is why the Client is responsible for managing the polygraph processes involved in a run, including commanding a Server to co-operate.

Now that the Master role exists, it would be cleaner for some of plumage_run's aggregation and control behaviour to be moved to it, leaving Client to act a lot more like Server as a temporary resource acting as a single component of a larger run without needing knowledge of it.

Machine resources could then potentially be offered as pools of multiple clients and servers, to allow multi-machine tests that scale beyond the capabilities of a single machine.

This would also pull host variables, and defining of client-server pairings, back toward the Master, centralizing configuration of them and making them easier to manage.