icgc-argo/workflow-graph-manager

Workflow Graph Manager

Workflow Graph Manager

Workflow Graph Manager

workflow-graph-manager's Issues

GQL Query Resolvers: Graph-manager --> Rabbit MQ Management API

Connect the graph manager api to the rabbit-management-api to be able to query running queues.

Expected Outcome

At https://workflow-graph-manager.rdpc-dev.cancercollaboratory.org/graphql, you can:

• Queue entity is queryable in GraphQl
• Messages are are queryable (graphEvent AND graphRun) entity in GraphQL

Evaluate Rabbit MQ Management API

• make sure the provided API can complete the queries we need
• if not, come up with a contingency solution that would allow us to interact with Rabbit MQ
• investigate whether this is deployed in our current environment.

https://www.rabbitmq.com/management.html

Graph Manager API

Build an API to interact with the WF Graph

https://wiki.oicr.on.ca/display/icgcargotech/Graph+Manager+Design%3A+RabbitMQ+Reporting+and+Queue+Management

Steps:

1. Logging
2. Queuing and Messaging

• we need to be able to read to the RabitMQ queues (e.g. on an align node, how many WFS are queued and running)
• we need to be able to manipulate in som circumstanded the state of a message (i.e move to back of queue, Move to front of queue, Cancel)

3. Node
4. Pipeline

GQL Mutations Resolvers: Graph-manager --> Rabbit MQ Management API

Expected Outcome

From https://workflow-graph-manager.rdpc-dev.cancercollaboratory.org/graphql, see mutations for:

• Message in Start queue
• Move to back of queue
• Move to front of queue
• Cancel

GQL Query Resolvers: Pipeline and Node entities

The Graph Manager should talk to the Kubernetes API to get information on:

• Pipelines
• Nodes
• Queue Names

This is a query only, no actions (mutation). This is will be done in a near real-time manner (see wiki: https://wiki.oicr.on.ca/pages/viewpage.action?pageId=154539008)

Expected Outcome

• Pipeline entity exists in the GraphQL that can be queried against
• Nodes entity exists in the GraphQL that can be queried against

Investigation: K8s Custom Resource Definition

Investigate Kubernetes custom resource definition with controller/operator.

Why

The Pipeline entity will need to speak with a K8s API to deploy nodes and introspect on deployed nodes, we want to simplify the logic in graph manager in favour of leveraging as much of the tools in K8s as possible.

Expected Outcome

• determine if this is possible, work required, benefits
• provide CRD (custom resource definition) example

workflow-graph-manager: Migrate from Dockerhub to Github Packages

Graph Node Repository

• Connect to Kubernetes
• Reads in the pods with config labels
• Have it return the pipeline/node configuration labels as as "pipeline/node" DTO
• Tests included!

Sonar: Near-Real Time reporting of data

• Talks to the graph-node repository (#13) and queues and messages (#15) repository to update graph state at set time intervals
• Expose methods to interact with that state (ex. GET pipeline, GET nodes) used by GQL resolver.

Rabbit MQ API interaction repository

queues-and-messages repository

• Talks to Rabbit MQ Management API
• Abstract into the Graphs Queues and Messages data types

Scaffold Project

• make it real
• plan for realness: https://wiki.oicr.on.ca/pages/viewpage.action?spaceKey=icgcargotech&title=Graph+Manager+Design%3A+RabbitMQ+Reporting+and+Queue+Management

Expected Outcome

• Reactor all the things
• GQL server with Apollo federation
• Entities as defined in plan for realness
• Docker + Jenkins

Implement GraphLog

• Read logs from ElasticSearch
• Implement GQL Entity resolver
  • with filters for all entity fields in graph manager (pipeline, node, etc)

Deployment: Node labels for identification

Labels include for example:

• pipeline
• node

When we deploy an entity, it should

• all pods have the pipeline label

Expected Outcome

• Define the label contract for pipelines, nodes etc
• Define the Network Policy labels that must come with a pipeline
• When the helm chart is deployed, it has the correct label are defined