com-pas/compas-deployment

CoMPAS Deployment repository

compas-deployment's Issues

Create Robot Tests for functionality on the tab "CoMPAS Versions"

As a developer
I want to be sure all functions on the tab "CoMPAS Versions" are working
So that users won't get unexpected errors when using CoMPAS (OpenSCD).

Currently the basic tests are implemented to test CoMPAS OpenScd functionality.
We need to extend this more for the tab "CoMPAS Versions".

• Restore a version
• Add a version
• Delete a version
• Delete all versions
• Compare versions
• Compare current version

Deploy example CoMPAS files automatic to demo.compas.energy

As a CoMPAS demo user

I would like to see example SCL files in order to see the CoMPAS capabilities

Acceptance criteria:

Check if the files are in the repository once day, if removed, place them back

Background;
This can be done with a Lambda funtion in AWS

CoMPAS-public-examples.zip

Reverse Proxy container fails to build

There has been some update on the OpenResty container which cause the container build to fail for the Reverse proxy Image.

Step 7/12 : RUN luarocks install lua-resty-openidc
 ---> Running in bedc7a78fd87
Warning: Failed searching manifest: Failed downloading https://luarocks.org/manifest-5.1 - failed downloading https://luarocks.org/manifest-5.1
Warning: Failed searching manifest: Failed downloading https://raw.githubusercontent.com/rocks-moonscript-org/moonrocks-mirror/master/manifest-5.1 - failed downloading https://raw.githubusercontent.com/rocks-moonscript-org/moonrocks-mirror/master/manifest-5.1
Warning: Failed searching manifest: Failed downloading https://luafr.org/luarocks/manifest-5.1 - failed downloading https://luafr.org/luarocks/manifest-5.1

Error: No results matching query were found for Lua 5.1.
To check if it is available for other Lua versions, use --check-lua-versions.
The command '/bin/sh -c luarocks install lua-resty-openidc' returned a non-zero code: 1
ERROR: Service 'reverse-proxy' failed to build : Build failed

Redirect after login fails in Docker Compose

Subject of the issue

Often after login on the Docker Compose environment using KeyCloak the redirect fails and KeyCloak returns a error.

Your environment

Both Chrome and Firefox

Steps to reproduce

Tell us how to reproduce this issue.

• Go to the URL 'http://localhost'
• Login screen is shown, log in use the scl-data-editor user
• After successful login the browser is redirect and the following message is shown 'state from argument does not match state restored from session'

This is not always happening, but if the login was successful, restart the docker-compose environment and try it again. In this cases it mostly went wrong for me during second time trying to login.

Expected behaviour

After login is successful being redirected to the start page of OpenSCD.

Actual behaviour

Error page show, going to http://localhost/ again most of the time solves the issue and the start page is shown.

Add SCL Validator to Docker Compose

Add the new SCL Validator service to the Docker Compose files.

• Also mount a directory to the volume which will hold the Custom OCL Files/NSDoc files.
• Add this directory to the Git Ignore list preventing commiting any files to GitHub.
• Update the instructions to explain how to add your own Custom OCL Files/NSDoc files in Docker Compose.

Also create a Robot Test to check if the validation is working through OpenSCD.

Authentification error

Hi,

I try to login on the compas OpenScd-Version.

Sadly it is not possible for me to get authenticated with keycloak
using
Username: user
PW: user

Following error appear :

May you help me?

Create a Robot Test for checking error responses

Create a Robot Tests to check the error response

• For the Auto Alignment Service when a invalid XML is passed, this can be something simple like a incomplete diagram where there are no Terminals defined.;
• For the CIM Mapping Service when a invalid XML is passed;
• For the Data Service when a invalid name tes/*.ssd. is used to save a new SCL File;

In the Log Window of OpenSCD there should be a line indicating that something has gone wrong.

Make Docker Compose setup stable

As a developer
I want the Docker Compose to work always
So that I don't have to try it more then once

Background:
After starting docker compose and going to localhost the login is always shown.
But after the logging OpenSCD not always starts, there are loading errors of all JS and so files.
We need to investigate why this is happening.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update postgres docker tag to v16.3

Detected dependencies

docker-compose

compas/docker-compose-postgresql.yml

• postgres 16.2
• lfenergy/compas-scl-data-service 0.15.0-postgresql
• lfenergy/compas-cim-mapping 0.10.4
• lfenergy/compas-scl-auto-alignment 0.5.1
• lfenergy/compas-scl-validator 0.6.1
• lfenergy/compas-open-scd v0.33.0.7

---

• Check this box to trigger a request for Renovate to run again on this repository

Automatic SSL certificate renewal

User Story

As a CoMPAS community

I want to automatic renew the certificates on demo.compas.energy

So that my SSL certificate always work

Acceptance criteria

Automatic renewable

Background:
https://docs.aws.amazon.com/acm/latest/userguide/domain-ownership-validation.html

Back to latest alpine-fat image for Reverse Proxy

There was a version of "openresty/openresty:alpine-fat" (openresty/openresty:alpine-fat) that failed, see #68.

But it would be nice to check if this image version will work again in the future and return back to this version.

CoMPAS with the latest Docker compose (v2)

Subject of the issue

Check if the the deployment is still running with the latest docker compose. Docker compose v2.

Steps to reproduce

Deploy with docker compose v2.

Expected behaviour

Software should run

Actual behaviour

The docker looks started.
When checking the status, the dockers are doing down.

Solution:
Modify https://github.com/com-pas/compas-deployment/blob/main/README.md and add intructions for V1 and V2.

Acceptance criteria:
Solutions that will work for the users;
Updating the documentation might already be enough.

Include task definition for CoMPAS Sitipe Service

Subject of the issue

Currently we have task definition for these services
CoMPAS-Cim-Mapping
CoMPAS-OpenSCD
CoMPAS-Scl-Auto-Alignment
CoMPAS-Scl-Data-Service

but not for CoMPAS-Sitipe-Service

Therefore, our aws environment does not have an instance of this service and we have some breaking behavior in demo.compas.energy

Create Github action for AWS deployment

User Story

As a maintainer

I want to deploy new CoMPAS versions to AWS in an automatic way

So that demo.compas.energy stays current with the development of CoMPAS.

Acceptance criteria

Github action (with at least 2 steps)

To do
Create a new AWS task definition withe the latest version
Deploy the task definition

** Background **

https://github.com/aws-actions/amazon-ecs-deploy-task-definition#task-definition-file

Test "Open project from CoMPAS (Rest) and Save to Local File" is failing in ci

TestCase001-04 :: Open project from CoMPAS (Rest) and Save to Loca... | FAIL |
TimeoutError: locator.click: Timeout 10000ms exceeded.
=========================== logs ===========================
waiting for locator('mwc-drawer[id="menu"] mwc-icon-button[label="Menu"] > button')
  locator resolved to <button aria-label="menu" class="mdc-icon-button">…</button>
attempting click action
  waiting for element to be visible, enabled and stable
  element is visible, enabled and stable
  scrolling into view if needed
  done scrolling
  <span slot="title">…</span> intercepts pointer events
retrying click action, attempt #1
  waiting for element to be visible, enabled and stable
  element is visible, enabled and stable
  scrolling into view if needed
  done scrolling
  <span slot="title">…</span> intercepts pointer events
retrying click action, attempt #2
  waiting 20ms
    [ Message content over the limit has been removed. ]
  element is visible, enabled and stable
  scrolling into view if needed
  done scrolling
  <span slot="title">…</span> intercepts pointer events
retrying click action, attempt #17
  waiting 500ms
  waiting for element to be visible, enabled and stable
  element is visible, enabled and stable
  scrolling into view if needed
  done scrolling
  <span slot="title">…</span> intercepts pointer events
retrying click action, attempt #18
  waiting 500ms
  waiting for element to be visible, enabled and stable
  element is visible, enabled and stable
  scrolling into view if needed
  done scrolling
  <span slot="title
Tip: Use "Set Browser Timeout" for increasing the timeout or double check your locator as the targeted element(s) couldn't be found.

Possible fix: From the output it seems like adding a Sleep before executing one of the three "Open Menu" tasks in the TestCase could help fixing this error

Fork repository https://github.com/ppodgorsek/docker-robot-framework

Maybe it's better to make a fork of the repository https://github.com/ppodgorsek/docker-robot-framework, so that we can maintain the version our self. Currently they are already not up-to-date and maybe is causing the chromium error.

Add template generator to CoMPAS distrobution

As an demo user,
I would like to have template generator plug-in into CoMPAS
So I test the solution

Source:
https://github.com/ca-d/oscd-template-generator

The template generator can be seen here in action:
https://openenergytools.github.io/scl-editor/

Create Robot Tests for menu 'Save project as' and 'Save as version'

User Story

Create Robot Tests to test the new menu options 'Save project as' and 'Save as version' in combination with the Data Service.

Acceptance criteria

Test covering the two new menu options.

Docker Compose for CoMPAS application

Create a Docker Compose file for running multiple :

• Running Keycloak
• Running SCL Data Service
• Running BaseX
• Running OpenSCD
• Running CIM Mapping

Acceptance criteria:
Create a Docker Compose, store it in this repository.

Fix skipped e2e tests

As a compas developer I would like to delete unhelpful skipped tests and continue executing the valid tests that are being skipped right now, either by using Robot Framework or whatever other technology we decide to use. This would improve our quality and trust in the test suites that we run.

Acceptance Criteria:

• Tests not related to SCL Validator are not skipped and pass successfully.

Apple Silicon Support

User Story

As a developer with an Apple Silicon (M1) laptop

I want to run CoMPAS locally

So that I can try it out and eventually work on it

Acceptance criteria

• I can start and use CoMPAS on an Apple Silicon Laptop

Hints

Create Robot Tests for CIM Mapping functionality

As a developer
I want to be sure the CIM Mapping is working
So that users won't get unexpected errors when using CoMPAS (OpenSCD).

Currently the basic tests are implemented to test CoMPAS OpenScd functionality.
We need to add a/some test(s) for the CIM Mapping.

• Create project using CIM Mapping

Feat: Create single docker-compose file with profiles (for postgres and basex

Migrate different docker-compose files to a singular docker-compose file that's using profiles (https://docs.docker.com/compose/profiles/)

Deployment doesn't run locally

When cloning the compas-deployment repository and building the docker containers according to the dev document I can start the login page with localhost (port 80). After filling in the credentials Im getting the following error:
accessing token endpoint (http://localhost/auth/realms/compas/protocol/openid-connect/token) failed: localhost could not be resolved (3: Host not found).

I’ve tested the token request with Postman and can confirm that keycloak works correctly. My guess right now is that the DNS resolvement doesn’t go correctly inside the docker container (that localhost can not be resolved inside the container).

Acceptance criteria

• Make sure the deployment works locally.

Add Robot Test for SCL Auto Alignment menu

As a developer
I want a integration test for the SCL Auto Alignment
So that the integration between OpenSCD and SCL Auto Alignment is working.

At least build a test to execute the Menu Item on a specific SCD.
Maybe also check if the result is a little bit okay.

Add SCL Auto Alignment Service to Docker Compose

As a developer
I want to update test setup (Docker Compose)
So that it include the SCL Auto Alignment Service and the latest OpenSCD version using the Auto Alignment Service.

Create Docker Compose version of PostgreSQL implementation

As a user of CoMPAS
I want to be able to test the application using PostgreSQL as database
So that I have a reference implementation of the PostgreSQL version

Background:
Use the Docker Image that implements the PostgreSQL Repository of the Scl Data Service. (com-pas/compas-scl-data-service#126)
We can copy the Docker Compose file of the BaseX version and replace the BaseX Database with a PostgreSQL Database.

Split up sitipe task definition into 2 separate definitions

Currently the sitipe task definition contains the mssql database and the sitipe service.

When the sitipe service is updated, is results in a new deployment, which means that the mssql database is cleared. If we split up the task definitions into 2 separate definitions, the database will stay intact on a redeploy of the sitipe service.

Setup Renovate for docker-compose

Since Dependabot doesn't support Docker-Compose files, we need to have a different provider that supports Docker-Compose files.
Luckily, Renovate (OpenSource) does.

We can have Renovate next to the current Dependabot to only look into the Docker-Compose file and create a PR whenever a newer version of a docker image is released.

https://docs.renovatebot.com/modules/manager/docker-compose/

https://developer.mend.io/ is used; it is connected to the organisation;
Renovate will not create pull-request that are already made by dependabot;

Acceptance criteria:
Renovate will creat pull-requests when their is a new image in dockerhub for one of the services

Redesign the Keycloak setup

As a Developer
I want to have the Keycloak setup with OpenSCD changed
So that OpenSCD has less knowledge about KeyCloak

Background:
Now KeyCloak is integrated in OpenSCD using OpenID, but not all environments work like this, like Alliander.
We protected the whole applicatie using a SSO, so when you browse to OpenSCD you first need to login.

Solution:

• Configure the Reverse Proxy to protected the URl's using KeyCloak and the login is not needed anymore in OpenSCD.
• And the dependency with KeyCloak is also not needed anymore in OpenSCD.
• Also added the Bearer is not needed anymore, because the Reverse Proxy will do this.

Acceptance:

• When browsing to the CoMPAS OpenSCD application you first need to login before OpenSCD is displayed.
• Also the injection of the JWT Bearer is done in the Reverse Proxy.

Add backdoor for local development environment

User Story

As a Developer

I want to have a backdoor in the reverse proxy to have my open-scd instance running locally outside of docker

So that it improves my development speed

Acceptance criteria

• CoMPAS OpenSCD must be proxied by the reverse proxy (nginx).
• Frontend must be able to access the backend (Check cors + authentication)

Security Policy violation SECURITY.md

This issue was automatically created by Allstar.

Security Policy Violation
Security policy not enabled.
A SECURITY.md file can give users information about what constitutes a vulnerability and how to report one securely so that information about a bug is not publicly visible. Examples of secure reporting methods include using an issue tracker with private issue support, or encrypted email with a published key.

To fix this, add a SECURITY.md file that explains how to handle vulnerabilities found in your repository. Go to https://github.com/com-pas/compas-deployment/security/policy to enable.

For more information, see https://docs.github.com/en/code-security/getting-started/adding-a-security-policy-to-your-repository.

---

This issue will auto resolve when the policy is in compliance.

Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.

Create reverse proxy for CoMPAS services

In order to make OpenSCD work for example (with the CORS problems), it's a good practice to have a reverse proxy in front of our services.

Setup Release-Please

To make easier releases, we should set up the GitHub Action Release please.
See OpenSCD Release-please for more info

Improve release process

In order to share new functionality we need to communicate that to the public.

Reasons to share CoMPAS suite releases:

• Services depend on each other (e.g. some services might not be compatible)
• Not all end-users are interested in seperate updates (if they are, they can always do that)

How to share a release:

• Create a Github action to automaticly get the release notes from all services updates; Store this in the github tag and make a github release
• Write some manual release notes
• Update changelog.md to refer to the github release
• Automate plublications (e.g. Linkedin, slack etc)

Idea:

NGINX to TRAEFIK

Mover NGINX to TRAEFIK in order to:

Allow local testing

Fix cachings issues (dynamic load)

Development speed (is hard to run the front-end

NGINX is currently used in

Docker-compose on Github.