opdev/fips-assessments

fips-assessments's Issues

Add ansible-automation-platform operator to fips-assessments tasks

Add Ansible tasks that can create the following CR(s) and validate non-failed status: https://github.com/opdev/fips-assessments/blob/main/ansible-automation-platform

Implement fips-assessments tests for OpenShift GitOps

The steps to implement the testing are:

Create the tasks according to the check.txt
Handle any necessary yaml
Generate results to be reported back to opcap-ansible

opcap-ansible integration will be treated on opdev/opcap-ansible#18

Add rhacs-operator to fips-assessments tasks

Add Ansible tasks that can create the following CR(s) and validate non-failed status: https://github.com/opdev/fips-assessments/blob/main/rhacs-operator

Can't download published fips-assessment collection from private automation hub

We are not able to download the published fips-assessment collection from our private automation hub.

Attempting to download the collection tarball via UI results in hang with no response.

Attempting to download via ansible-galaxy collection install results in following:

Starting collection install process
Fetching a collection tarball for 'opdev.fips_assessments:0.0.1+b437632' from Ansible Galaxy
Downloading https://opdev-hub-aap.apps.eng.opdev.io/api/galaxy/v3/plugin/ansible/content/published/collections/artifacts/opdev-fips_assessments-0.0.1+b437632.tar.gz to /Users/<username>/.ansible/tmp/ansible-local-701613ze3ms4z/tmp1wydh9xg/opdev-fips_assessments-0.0.1+b437632-o755pn83
ERROR! Failed to download collection tar from 'published' due to the following unforeseen error: <urlopen error [Errno 8] nodename nor servname provided, or not known>. <urlopen error [Errno 8] nodename nor servname provided, or not known>

Troubleshooting via curl -v -L -u <username>:<password> https://opdev-hub-aap.apps.eng.opdev.io/api/galaxy/v3/plugin/ansible/content/published/colle[…]artifacts/opdev-fips_assessments-0.0.1+b437632.tar.gz reveals multiple redirects that end up in Could not resolve host: aap-hub-hl.aap.svc.cluster.local

The fix may be to update PULP_ANSIBLE_CONTENT_HOSTNAME in /etc/pulp/settings.py with resolvable url. We should be able to do this via kind: AutomationHub spec.

Determine workflow for publishing fips-assessment collection to private automation hub

Our plan is to automate the testing/publishing of fips-assessment collections to private automation hub (PAH) upon any changes to this upstream repo.

We can also add newly approved collections to our execution environment image for ease of use when executing reports.

Right now, our team cluster's PAH is currently configured with SSO. As a result, PAH api tokens only live for 1-day. This token is needed to publish or install a collection to/from PAH via ansible-galaxy cli tool.

We will find a way to use a long-lived token so we can automate this process with no intervention.

A note that we will also need to update the primary playbooks to import opdev.fips_assessments.

Add 3Scale to fips-assessments tasks

Add Ansible tasks that can perform the following checks: https://github.com/opdev/fips-assessments/blob/main/3scale/check.txt

Note: It will require https://github.com/opdev/fips-assessments/blob/main/3scale/setup.txt

opcap-ansible integration will be treated on opdev/opcap-ansible#23

Add eap operator to fips-assessments tasks

Add Ansible tasks that can create the following CR(s) and validate non-failed status: https://github.com/opdev/fips-assessments/blob/main/eap-operator

Need to add ansible-lint to requirements.txt

Add ansible-lint to requirements.txt so we can utilize it when doing dev within python virtual env

Datagrid assessment could pass a false positive

First, it uses the old "failed" term instead of just "fail".

Second, If an earlier task fails, but a subsequent task succeeds, it will overwrite the result, possibly resulting in a false positive.

Need to add ansible-lint as a GH action

Ansible-lint will help to keep the code lint-free.

Add OpenShift-Pipelines to fips-assessments tasks

Add Ansible tasks that can perform the following checks: https://github.com/opdev/fips-assessments/blob/main/openshift-pipelines/check.txt

opcap-ansible integration will be treated on opdev/opcap-ansible#22

Add advanced-cluster-management operator to fips-assessments tasks

Add Ansible tasks that can create the following CR(s) and validate non-failed status: https://github.com/opdev/fips-assessments/blob/main/advanced-cluster-management

Implement openshift-serverless testing with opcap-ansible

The steps to implement the testing are:

• Create the tasks according to the check.txt
• Handle any necessary yaml
• Generate results to be reported back to opcap-ansible

opcap-ansible integration will be treated on its own issue and repo.

Galaxy import warning message: Could not get role description, no role metadata found

Warnings show up when uploading the collection to our PAH -

"messages": [
        {
            "time": 1697474847.7705119,
            "level": "INFO",
            "message": "Importing with galaxy-importer 0.4.13"
        },
        {
            "time": 1697474847.823006,
            "level": "INFO",
            "message": "Getting doc strings via ansible-doc"
        },
        {
            "time": 1697474847.8272913,
            "level": "INFO",
            "message": "Finding content inside collection"
        },
        {
            "time": 1697474847.8307352,
            "level": "INFO",
            "message": "Loading role datagrid"
        },
        {
            "time": 1697474847.8337407,
            "level": "WARNING",
            "message": "Could not get role description, no role metadata found"
        },
        {
            "time": 1697474847.8617287,
            "level": "INFO",
            "message": "Loading role eap"
        },
        {
            "time": 1697474847.8739657,
            "level": "WARNING",
            "message": "Could not get role description, no role metadata found"
        },
        {
            "time": 1697474847.8788185,
            "level": "INFO",
            "message": "Loading role serverless_operator"
        },
        {
            "time": 1697474847.882334,
            "level": "WARNING",
            "message": "Could not get role description, no role metadata found"
        },
        {
            "time": 1697474847.8865845,
            "level": "INFO",
            "message": "Loading role threescale_operator"
        },
        {
            "time": 1697474847.8898234,
            "level": "WARNING",
            "message": "Could not get role description, no role metadata found"
        },
        {
            "time": 1697474847.8939724,
            "level": "INFO",
            "message": "Loading role openshift_pipelines_operator_rh"
        },
        {
            "time": 1697474847.8973284,
            "level": "WARNING",
            "message": "Could not get role description, no role metadata found"
        },
        {
            "time": 1697474847.903375,
            "level": "INFO",
            "message": "Loading role rhsso_operator"
        },
        {
            "time": 1697474847.9070375,
            "level": "WARNING",
            "message": "Could not get role description, no role metadata found"
        },
        {
            "time": 1697474847.9112422,
            "level": "INFO",
            "message": "Loading role advanced_cluster_management"
        },
        {
            "time": 1697474847.9147143,
            "level": "WARNING",
            "message": "Could not get role description, no role metadata found"
        },
        {
            "time": 1697474847.9219549,
            "level": "INFO",
            "message": "Loading role devspaces"
        },
        {
            "time": 1697474847.9254007,
            "level": "WARNING",
            "message": "Could not get role description, no role metadata found"
        },
        {
            "time": 1697474847.9297173,
            "level": "INFO",
            "message": "Loading role rhacs_operator"
        },
        {
            "time": 1697474847.9329712,
            "level": "WARNING",
            "message": "Could not get role description, no role metadata found"
        },
        {
            "time": 1697474847.9372487,
            "level": "INFO",
            "message": "Loading role amq_streams"
        },
        {
            "time": 1697474847.9407094,
            "level": "WARNING",
            "message": "Could not get role description, no role metadata found"
        },
        {
            "time": 1697474847.9469233,
            "level": "INFO",
            "message": "Loading role ansible_automation_platform_operator"
        },
        {
            "time": 1697474847.9503584,
            "level": "WARNING",
            "message": "Could not get role description, no role metadata found"
        },
        {
            "time": 1697474847.9545233,
            "level": "INFO",
            "message": "Loading role openshift_gitops_operator"
        },
        {
            "time": 1697474847.957911,
            "level": "WARNING",
            "message": "Could not get role description, no role metadata found"

Add devspaces operator to fips-assessments tasks

Add Ansible tasks that can create the following CR(s) and validate non-failed status: https://github.com/opdev/fips-assessments/blob/main/devspaces

3Scale Operator tasks attempt to install CRD that isn't part of 3Scale

3Scale Operator Ansible tasks/checks attempts to install APICast CRD which is not part of the 3scale-operator bundle. It appears that this CRD is from apicast-operator.

Add amq-streams operator to fips-assessments tasks

Add Ansible tasks that can create the following CR(s) and validate non-failed status: https://github.com/opdev/fips-assessments/blob/main/amq-streams

Add data-grid to fips-assessments tasks

Add Ansible tasks that can perform the following checks: https://github.com/opdev/fips-assessments/blob/main/data-grid/check.txt

opcap-ansible integration will be treated on opdev/opcap-ansible#24

Revise GitHub Actions workflow so that we don't publish to PAH

To simplify some encountered roadblocks with secrets and github actions/workflows when storing fips-assessments collection in our private automation hub, we can stop publishing to PAH and directly refer to the fips-assessments collection residing in this github repo in requirements.yml:

collections:
  - name: https://github.com/opdev/fips-assessments.git
    type: git
    version: main
...

@itroyano appreciate your efforts on https://github.com/opdev/fips-assessments/blob/main/.github/workflows/publish-collection.yml but we can eliminate this for now - unless you see an issue with the proposition.

Add rhsso-operator to fips-assessments tasks

Add Ansible tasks that can create the following CR(s) and validate non-failed status: https://github.com/opdev/fips-assessments/blob/main/rhsso-operator

Add openshift-data-foundation operator to fips-assessments tasks

Add Ansible tasks that can create the following CR(s) and validate non-failed status: https://github.com/opdev/fips-assessments/blob/main/openshift-data-foundation

Add amazon.aws to galaxy dependencies

Having the dependency in requirements.yml is not enough for amazon.aws to be pulled in when the fips-assessment collection is installed. It needs to be added to galaxy.yml.