ana-cc/monroe-cli

Installing in a Debian 9 VM

The following will install the latest version of the cli:

apt install git python3-dev python3-setuptools build-essential libffi-dev libssl-dev python3-straight.plugin python3-cryptography
git clone https://github.com/ana-cc/monroe-cli.git
cd monroe-cli
python3 setup.py develop

To set up your certificate, copy your pkcs12 certificate to the new VM, and run:

monroe setup <certificate-file>

Notes:

The tool generates an ssh key to use for ssh-ing into the nodes, it asks the user to proceed with the key generation it can't find one in ~/.monroe

The tool depends on having crt and key files for an experimenter certificate in ~/.monroe/. This is automated by running monroe setup <cert.p12> on a fed4fire pkcs12 certificate, which creates a .monroe folder in your home directory, and extracts the cert and key in pem format.

Library:

The library can be imported as such: from monroe.core import *

The public funtions are all documented and it should be straightforward to use. The following example creates a scheduler, adds an experiment, modifies it and submits it:

s = Scheduler("/home/user/.monroe/mnrCrt.pem", "/home/user/.monroe/mnrKey.pem")
e = s.new_experiment()
e.script('user/dockercontainer')
e.name('myexperiment')
e.countries('Norway,VTAB)
e.nodecount('20')
s.submit_experiment(e)

The following example creates a scheduler, and retrieves list of nodes in a particular country:

s = Scheduler("/home/user/.monroe/mnrCrt.pem", "/home/user/.monroe/mnrKey.pem")
nodelist = [node.id() for node in s.nodes() if node.site() == 'spain']

monroe-cli's Issues

wishlist: package cli as a docker container

For easier usage, it would be interesting to package the tool as a docker container, and avoiding the need of locally installing the extra package dependencies.

Mockup invocation

docker run monroe/cli create my/experiment --new --ssh

This makes extra sense since experimenters are already using docker to create the experiment containers, the monroe/base image is therefore already downloaded, etc.

I'll see if I go about it myself, but for now I'm just adding it to the wishlist issues.

It should be straightforward, but we need to figure out the mapping of ~/.monroe directory and the cert setup.

possible parsing problem with jsonstr

Using lists of lists seems to produce incorrect quoting when the json is passed:

monroe create docker.monroe-system.eu/monroe/monroe-web/image --jsonstr "http_protocols":["h1","h1s"]
"urls":[["https://www.httpvshttps.com/","http://www.kau.se"],
["http://www.kau.se","http://www.kau.se"]]

Allocated task 15677.
Additional options passed: {'http_protocols': ['h1', 'h1s'],
'urls': ['https://www.httpvshttps.com/', 'http://www.kau.se]', '[http://www.kau.se', 'http://www.kau.se']}

[ is included in the quoted string for the second list, and also the overarching [] are removed.

wishlist: display running/scheduled tasks for nodes

Sometimes there are no available nodes for scheduling, and it would be interesting to know what experiments are currently running, for how much longer, or when the next scheduled job starts. From a maintenance perspective to find out what experiments are currently active (maybe someone has allocated 3 hours for all testing nodes?). From a user perspective, getting some indication when I would be able to run my next experiment?

Mockup

monroe resources  [--countries , --nodes, --new, --old, --deployed]
Node 45: Running docker.monroe-system.eu/deployed/webworks for 34 of 180 allocated minutes (146 remaining). Next schedule in 4 hours 32 minutes (Maintenance hour)
Node 46: Idle. Next schedule in 1 hour 30 minutes (steven76/mycontainer)

wishlist: check cert hash (md5 vs sha1)

When using the scheduler certificate, it would be convenient the tool would warn of incompatible md5 hash and recommend the course of action (update iminds password, get new cert, submit fingerprint).

Alternatively, configure openssl with tls-cipher "DEFAULT:@SECLEVEL=0", but the security implications of doing this should be considered (there's a reason why md5 is disabled...).

commandline checking can be done using

openssl x509 -text -in ~/.monroe/mnrCrt.pem

and find the "Signature Algorithm" line.

invalid int value for --nodes argument

$ monroe create --nodes 400 steven76/dennis
usage: monroe create [-h] [--name NAME] [--deployed] [--nodecount NODECOUNT]
                    [--ifcount IFCOUNT] [--maxnodes] [--duration DURATION]
                    [--start START] [--traffic TRAFFIC] [--storage STORAGE]
                    [--logfile LOGFILE] [--ssh] [--new] [--old]
                    [--jsonstr JSONSTR [JSONSTR ...]]
                    [--countries COUNTRIES [COUNTRIES ...]]
                    [--nodes NODES [NODES ...]]
                    [--recurrence <period, finish time> <period, finish time>]
                    [--availability]
                    script
monroe create: error: argument --nodes: invalid int value: 'steven76/dennis'

Using --nodes=400 works however, but does not allow for multiple nodes to be specified.
The argument parsing for --countries is similar, so check if this also needs adjustment.

Upgrade core.py to use python-requests

As the user certificates are now sha1, the code can be upgraded to use python-requests. The documentation should also be changed as the tool will not be restricted to Debian VMs anymore.

--cert argument unrecognized

$ git rev-parse --short HEAD
1fbd904
$ monroe whoami
Please run monroe setup --cert <certificate> to be able to submit experiments and retrieve results.
$ monroe setup --cert iminds-alfredsson.p12
usage: monroe [-h] Command ...
monroe: error: unrecognized arguments: --cert
$

It works when excluding --cert.

$ monroe setup iminds-alfredsson.p12
Enter passphrase:
Your certificate files were stored in ~/.monroe
$ monroe whoami
Authentication ID: **, Name: ******, Storage Quota remaining: ******* bytes
$

python3-straight.plugin not in ubuntu 16.04 LTS repos (remove?)

It seems that package python3-straight.plugin is in the repo of only in the latest version of ubuntu and not in the repo of the current LTS

https://packages.ubuntu.com/bionic/python3-straight.plugin

By going through the code it seems to me that it is not really needed, since the plugins folder is empty.
Would it make sense to remove this dependency?