openflighthpc/concertim-cluster-builder

This repos is for managing the cluster builder for the concertim cluster portal project

concertim-cluster-builder's Issues

Include cluster name when creating order

As well as the openstack ID that is currently provided, we should include the cluster's name when creating an order so this can be more easily included in invoices.

Clusters vs Rack creation - Order ID mismatch

When using heat, this directly creates a stack (see http://10.151.0.184/project/stacks/). However, when using magnum and sahara, this creates a cluster (see http://10.151.0.184/project/clusters), which in turn contains a stack.

When cluster builder creates an order in the middleware service, the cluster id is being sent for magnum (and presumably sahara, though I've not been able to test this). And for heat, the stack id is being sent.

However the middleware service assumes the stack id has been used for all order records. This means that for magnum/sahara stacks, creating racks in visualiser fails, as the middleware can't find an order with their stack id.

Ideally we should be consistent and always pass the stack id instead of the cluster id. It may be possible to obtain the stack id when creating such clusters, e.g. using magnum_client.clusters.get(cluster_id).stack_id, but it seems that this is None when the cluster is first created - further investigation is needed.

Alternatively we could update the middleware service to check if a stack has a cluster id - if present, using that instead to check an order, but this will make the process less client agnostic.

More generally it may be worth considering if there is any fundamental difference between a cluster containing a rack and just a rack on its own that needs to be differentiated in visualiser/elsewhere in concertim.

Address limitations with sahara cluster type variety

There are currently some limitations with the sahara cluster type variety:

For the direct launching of sahara clusters:

1. requires a network to be specified. We currently only have a guarantee that the user has access to the public1 network. Launching a sahara cluster on the public1 network is not a good idea. Ideally, we'd have some mechanism of ensuring that the user has non-external network available to them. There are several forms that could take: require the ops team (or similar) to create a single network and ensure it is available to every concertim user. Have the middleware configured to automatically make a certain non-external network available to each new concertim user. Have the middleware create a dedicated non-external network for each new concetim user.
2. requires that the network be specified by ID. Using the name of the network would provide a better experience for both the user and the cluster type author especially where defaults are concerned. It is possible to get the ID for a network from its name, but the exact incantation is currently unknown.

Indirect launching of sahara clusters via HOT template:

1. requires that the sahara plugin, plugin version, and image id are all provided by the user. It is possible to calculate these from the cluster template, but this is not currently done. This could be done by adding a new cluster type, say SaharaHeatWrapperHandler that is similar to SaharaHandler in how it determines values from the cluster template, but is also similar to HeatHandler in how it launches the cluster on openstack.
2. requires that the image and cluster template are provided by id not name. This could (perhaps?) be fixed with a better HOT template, or by using the SaharaHeatWrapperHandler, alluded to above, to perform the translation.

Currently, support for sahara clusters is low priority so this issue mostly exists to document these as known issues and suggest potential solutions.

Specify name property for volumes in resource groups

Currently if there are multiple volumes in a resource group, they are given the same name (e.g. node-vol). We should update this to give them unique, descriptive names as we do with servers in a resource group. e.g. node01-vol, node02-vol, etc.