openflighthpc / concertim-ansible-playbook Goto Github PK

View Code? Open in Web Editor NEW

0.0 3.0 0.0 85.18 MB

Ansible playbook for building a Concertim appliance

License: Eclipse Public License 2.0

Shell 84.94% Dockerfile 15.06%

concertim-ansible-playbook's Introduction

Concertim Ansible Playbook

Concertim Ansible Playbook deploys Alces Concertim services and infrastructure components in Docker containers.

Getting Started

You are viewing the development version of Alces Concertim Ansible Playbook. To deploy this version of Alces Concertim follow these instructions.

To deploy a released version of Alces Concertim select the tag for that release and follow the deployment instructions there.

Concertim services

Concertim Ansible Playbook deploys containers for the following Concertim projects:

Infrastructure components

Concertim Ansible Playbook deploys containers for the following infrastructure components:

Kill Bill an open source billing and payments platform.
MariaDB for MySQL database.
Nginx a HTTP and reverse proxy server.
Postgresql an open source object-relational database system.

Directories

ansible - Contains an Ansible playbook to deploy Concertim services and infrastructure components in Docker containers.
contrib - Contains a Vagrant development environment for developing the playbook in the ansible directory. Also contains an alternate Ansible playbook that can be used to create a development environment for the Concertim Components.
docs - Contains documentation

Developing

For details on how to develop and test the ansible deployment playbook see docs/DEVELOPMENT.md.

If you are looking for details on how to develop the Concertim services themselves see the contrib/dev/README.md.

Contributing

Fork the project. Make your feature addition or bug fix. Send a pull request. Bonus points for topic branches.

Read CONTRIBUTING.md for more details.

Copyright and License

Eclipse Public License 2.0, see LICENSE.txt for details.

This program and the accompanying materials are made available under the terms of the Eclipse Public License 2.0 which is available at https://www.eclipse.org/legal/epl-2.0, or alternative license terms made available by Alces Flight Ltd - please direct inquiries about licensing to [email protected].

Concertim Ansible Playbook is distributed in the hope that it will be useful, but WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OR CONDITIONS OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. See the Eclipse Public License 2.0 for more details.

concertim-ansible-playbook's People

Contributors

Watchers

concertim-ansible-playbook's Issues

How do we want to handle SSL certificates for MIA?

Previously, Concurrent/Concertim used self-signed certificates. The full chain for the certificate was installed on the MIAs/ISLAs allowing internal communication to be validated. Customers were required to either install the certificate authority(?) or simply accept the SSL validation warning.

Currently, we have MIA use the default SSL certificate on the machine. This is self-signed causing SSL validation issues, requiring the user to accept the certificate.

Going forwards, we could use a similar mechanism as Concurrent/Concertim. Or we could use Let's Encrypt certificates.

Using a similar mechanism as before, requires that we have a secure machine somewhere on which we can create the certificates.

Let's Encrypt have a short life span, so we would need an mechanism to automatically renew them. Such mechanism should be easy to develop, we've already done so before for OpenFlight's flight-www, but it does require that the MIA is internet accessible. There may or may not be automatic solutions available that do not require an internet accessible MIA.

Improve experience when upgrading concertim deployment

When upgrading concertim et al there are two situations that could require the images to be rebuilt and the containers to be recreated.

The git repository for the service, e.g., visualiser or the middleware, has been updated.
The git repository for the ansible playbook, i.e., this repository, has been updated.

Currently, the playbook automatically detects if the service repository has been updated, and rebuilds the image/recreates the container appropriately. That is case (1) above is handled correctly.

The playbook does not currently, detect that the ansible playbook (this repo) has been updated, and does not automatically rebuild the image/recreate the containers. That is case (2) above is not automatically handled and requires user intervention.

The user intervention is as follows:

concertim_images=$(docker compose -f /opt/concertim/opt/docker/docker-compose.yml ps --all --format '{{.Image}}' | grep '^concertim-')
docker compose -f /opt/concertim/opt/docker/docker-compose.yml down
for i in ${concertim_images} ; do docker image rm $i ; done

With that done the playbook can be re-ran and will create new images appropriately.

For many upgrades, upgrading the playbook will go hand-in-hand with upgrading the services, however this is not necessarily the case. The most reliable way to ensure that the images / containers are rebuilt as necessary, would be to automate removing the existing containers and images. A more targetted approach that attempts to detect when this is necessary and thereby limit the down time might be desirable in the long term, but for now a simpler approach is preferable.

A new playbook should be added to this repo to remove the concertim containers and images, such that upgrading an installation is done as follows:

cd /opt/concertim/opt/ansible-playbook/ansible

ansible-playbook \
  --inventory inventory.ini \
  --extra-vars "gh_token=$GH_TOKEN" \
  --extra-vars @etc/globals.yaml \
  down.yml

ansible-playbook \
  --inventory inventory.ini \
  --extra-vars "gh_token=$GH_TOKEN" \
  --extra-vars @etc/globals.yaml \
  playbook.yml

or perhaps

cd /opt/concertim/opt/ansible-playbook/ansible

ansible-playbook \
  --inventory inventory.ini \
  --extra-vars "gh_token=$GH_TOKEN" \
  --extra-vars @etc/globals.yaml \
  upgrade.yml

Possible gotcha with static assets when upgrading a concertim deployment

On initial deployment a docker volume is created and automatically populated by docker with the contents of visualisers public/ directory. This volume is then shared with the proxy container to have nginx serve the static assets. There is a potential gotcha here when upgrading concertim.

If a new version of concertim has additional content under public/ say at public/images/irv/concertim/ is that additional content automatically copied to the volume too or do we need to manage this ourselves?

What about it content is removed in an upgrade? Is that removed automatically by docker?

Remove set-user-defaults.rb script

In my prior understanding of how concertim was going to be deployed these scripts made sense. However, my prior understanding turned out to be incorrect. Having a script which sets the admin user's password to the default that it is already set to in the database migration is not useful. We should remove the https://github.com/alces-flight/concertim-ansible-playbook/blob/main/ansible/roles/ct-visualisation-app/tasks/create_default_users.yml task.

Appliance dev role should automate encryption setup

See point (1) at https://github.com/alces-flight/concertim-ansible-playbook/tree/main/vagrant#virtual-machines for steps that need to be manually completed when configuring the virtual machine for appliance development. The appliance-dev role should perform these automatically. It will need to be careful to not override any existing values set here.

Vagrant dev workflow - asset changes often not being auto reloaded

I've had some inconsistent issues after using the vagrant workflow to build visualiser & set it to dev mode, where changes to css are not being auto reloaded.

I haven't been able to determine what makes it sometimes work and sometimes not. But the vast majority of the time it does not work. I've tried rebuilding multiple times and stopping and restarting visualiser. I've had some success by shutting down the VM, deleting assets/builds/application.css and then restarting, but this is not consistent. There are no errors reported in the ct-vis-app screen. Once it is working it seems to continue working until visualiser is stopped.

I'm not sure if this is due to recent changes to this repo or visualiser itself. This was first experienced on 19th October - I pulled from main on the afternoon of the 18th but I'm not sure how far behind my main branch was before that (/what commits were added, that could be the cause). I'm using dev-2023-10-17

Add monit support for emma and mia. Improve support for phoenix modules.

See comment #13 (comment) for description of issue and potential solution.

Useful links:

Also, see the postgresql.service and [email protected].

Fix issue with credentials and image updates

See this comment on #61 for details.