qmstr/module

Generic Quartermaster module

Quartermaster module template

Generic Quartermaster module template.

Build the image locally

make build

Get into its shell

make run

Run linters locally

make linters

module's Issues

Let the modules directly contact the DB whenever possible

Modules (i.e., builders, analyzers, and reporters) can contact (read and/or write) the graph database in two ways:

1. Through the Quartermaster orchestrator via the qmstr run ... wrapper (green arrow)
2. Directly (red arrow)

Contacting the graph database through the Orchestrator causes the latter to be responsible for database management, a task that it shouldn't be responsible for. Therefore, modules should contact the graph database directly (as it already happens for the JSON reporter, purple arrow).

The qmstr run ... wrapper will continue to exist as long as it's more convenient for us.

Integration tests

The builder should be at least able to communicate with the master/orchestrator.

Python and CMake builders do this with the qmstr run ... wrapper.
Maven builders do this by patching the root pom.xml file, injecting the Quartermaster Maven plugin.

A docker-compose.yml file can help to instantiate such an integration test, even though it'd be a manual one.

Build and push the image in CI

With github.com/docker/build-push-action.
Use the commit ID as the image tag.

Take inspiration out of this: please exclude the deployment step (only CI, no CD). You will need to set secrets.DOCKER_USERNAME and secrets.DOCKER_PASSWORD as GitHub secrets.

Fix linters errors

Last run errors are available here.
The super-linter can be also executed locally with:

make linters

Add basic module image

A basic (and empty) module image should:

1. Retrieve the Quartermaster source code in one of its early stages
   • At the moment this is only possible by cloning the repo, in the future, it'd use releases.
   • Repo URL/release number as a build argument.

Import message queue synchronization code

All modules need to synchronize with the master/orchestrator before starting their execution.
This is done with RabbitMQ, and the synchronization code is in qmstr/broker-test.

More specifically, the modules/base folder is the one that needs to be imported.

Add linters

All linters are enabled by default in github.com/github/super-linter.

Build fails

From the last GitHub Action run:

Error: buildx call failed with: error: failed to solve: rpc error: code = Unknown desc = executor failed running [/bin/sh -c apk update     &&     apk add     --no-cache     git=2.30.0-r0      ca-certificates=20191127-r5     tzdata=2020f-r0     &&     update-ca-certificates]: exit code: 1