bcgov/trust-over-ip-configurations

OpenShift build and deployment configurations for the Trust Over IP components.

Trust Over IP Configurations

This repository contains the openshift-developer-tools compatible OpenShift configurations to customize the builds and deployments of vc-authn-oidc for use with the Trust Over IP project.

For information on how to use these configurations with the openshift-developer-tools scripts please refer to the documentation; README.md.

Name	Purpose	Configuration	Dev	Test	Prod	Notes
Access to Audio		Agent Issuer-Admin API Issuer-Web	Agent: https://a2a-agent-dev.apps.silver.devops.gov.bc.ca/ Agent-Admin: https://a2a-agent-admin-dev.apps.silver.devops.gov.bc.ca/ Issuer-Web: https://a2a-issuer-dev.apps.silver.devops.gov.bc.ca/ Issuer-Admin: https://a2a-issuer-dev.apps.silver.devops.gov.bc.ca/	Agent: https://a2a-agent-test.apps.silver.devops.gov.bc.ca/ Agent-Admin: https://a2a-agent-admin-test.apps.silver.devops.gov.bc.ca/ Issuer-Web: https://a2a-issuer-test.apps.silver.devops.gov.bc.ca/ Issuer-Admin: https://a2a-issuer-test.apps.silver.devops.gov.bc.ca/
Access to Audio Non Rev		Issuer-Web	Issuer-Admin: https://a2a-nonrev-issuer-dev.apps.silver.devops.gov.bc.ca/	Issuer-Admin: https://a2a-nonrev-issuer-dev.apps.silver.devops.gov.bc.ca/
IDIM-SIT		Agent Wallet	Agent-Admin: https://idim-sit-agent-admin-dev.apps.silver.devops.gov.bc.ca/ Agent: https://idim-sit-agent-dev.apps.silver.devops.gov.bc.ca/
IDIM		Agent Wallet	Agent-Admin: https://idim-agent-admin-dev.apps.silver.devops.gov.bc.ca/ Agent: https://idim-agent-dev.apps.silver.devops.gov.bc.ca/
Law Society BC		Agent Wallet	Agent-Admin: https://lsbc-agent-admin-dev.apps.silver.devops.gov.bc.ca/ Agent: https://lsbc-agent-dev.apps.silver.devops.gov.bc.ca/	Agent-Admin: https://lsbc-agent-admin-test.apps.silver.devops.gov.bc.ca/ Agent: https://lsbc-agent-test.apps.silver.devops.gov.bc.ca/	Agent-Admin: https://lsbc-agent-admin.apps.silver.devops.gov.bc.ca/ Agent: https://lsbc-agent.apps.silver.devops.gov.bc.ca/
BCVCPilot		Issuer-Admin Issuer-Web API DB Agent	Issuer-Admin: https://bcvcpilot-issuer-admin-dev.apps.silver.devops.gov.bc.ca/ Issuer-Web: https://bcvcpilot-issuer-dev.apps.silver.devops.gov.bc.ca/ Agent: https://bcvcpilot-agent-dev.apps.silver.devops.gov.bc.ca/ Agent-Admin: https://bcvcpilot-agent-admin-dev.apps.silver.devops.gov.bc.ca/	Issuer-Admin: https://bcvcpilot-issuer-admin-test.apps.silver.devops.gov.bc.ca/ Issuer-Web: https://bcvcpilot-issuer-test.apps.silver.devops.gov.bc.ca/ Agent: https://bcvcpilot-agent-test.apps.silver.devops.gov.bc.ca/ Agent-Admin: https://bcvcpilot-agent-admin-test.apps.silver.devops.gov.bc.ca/	Issuer-Admin: https://bcvcpilot-issuer-admin.vonx.io/ Issuer-Web: https://bcvcpilot-issuer.vonx.io/ Agent: https://bcvcpilot-agent.vonx.io/ Agent-Admin: https://bcvcpilot-agent-admin.vonx.io

Getting Help or Reporting an Issue

To report bugs/issues/feature requests, please file an issue.

How to Contribute

If you would like to contribute, please see our CONTRIBUTING guidelines.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.

trust-over-ip-configurations's Issues

Consolidate VC-AuthN instances

Until now we have been running two instances of VC-Authn (one under the defaultprofile and one under therev` profile) to support access requests with both non-revocable and revocable credentials, due to how the agents (mobile and enterprise) used to handle the requests.

After meticulous testing last week, it appears that only one use - case (presenting a non-revocable credential to a proof with a revocation interval) - yields inconsistent results across wallets, however most of the wallets and scenarios behave as expected.

We should move forward and consolidate the vc-authn instances asap in order to limit the number of moving components in the project, and the potential causes of confusion.

Migrate to OCP4

Overview

A few profiles use issuer-kit code for their apps, in particular:

• a2a: this is a development issuer used to test other applications. Only dev and test deployments exist.
  As such, in addition to updating the templates, the agent wallets will need to be migrated to the target namespace in OCP4 to ensure a seamless transition.
• lsbc: issuer used by an external team (The Law Society of British Columbia). The wallet contents need to be moved to OCP4 for all deployments (dev/test/prod).
• prime: issuer used by an external team (MoH Prime). The wallet contents need to be moved to OCP4 for all deployments (dev/test/prod). Additionally, instances of visual-verifier are used and need to be migrated over to OCP4.
• prime-sandbox: same as prime, just a sandbox development agent (no test, no prod). Wallet contents need to be moved over to OCP4.
• tails: deployments of hosted instances of indy-tails-server. PVC contents need to be moved to OCP4.
• default and rev: instances of vc-authn-oidc. Content of both wallet and database need to be transferred over to OCP4.
• backup: backup-container (postgres) instance. Data needs to be migrated (it can help migrating wallets/databases as it is already backing them up). Needs to be aligned to the latest backup-container configurations in backup-container.

Templates

This is a list of templates to be updated, assigned to the relevant person.

@wadeking98:

• visual-verifier
• issuer-wallet
• issuer-db
• issuer-agent
• issuer-api
• issuer-web
• backup

@esune:

• agent-rev
• agent
• controller-rev
• controller
• database
• wallet
• tails-server

Profiles:

This is a list of profiles to be deployed to OCP4, once the templates have been updated, assigned to the relevant person.

@wadeking98:

• a2a
• lsbc
• prime
• prime-sandbox

@esune:

• default
• rev
• tails

New Client - MDS Test Client

We would like to add the VC-authn installation as a IDP to them MDS TEST Keycloak realm.

I'm familiar with the procedure, but we still need to share certain values. Please me know how/who to connect with.

Issuer agent instances being heavily throttled in Trust Over IP (e79518) environments

Several of the issuer agent instances are being throttled at between >35% to >50% on average. Review and adjust the CPU resource allocations, primarily the CPU limit to reduce or eliminate the throttling. The goal should be to reduce throttling to <25% on average. For production an even lower average may be desirable.

These metrics can be easily reviewed using the Namespace Monitoring dashboard available through Grafana in our new monitoring stack.

Affected instances:

• idim, idim-sit, idim-qa
• lsbc

BC VC Pilot -- what are the rules/options around reuse of an emailed token

We found it odd that an emailed token can be reused -- e.g. clicked on after issuance. Should that be supported?

I think the behaviour that would be nice to see:

• Once issued, an attempt to reuse the token goes to a page with configurable text.
• An admin can edit the record for the invitation and "uncheck" the "issued" flag -- in which case the user could reuse the token. For this to work, there would likely need to be a "resend email" option.

This is raised for discussion initially to see what behaviour is currently supported so we can decide what we want to do. Please let us know what is currently supported.

@cvarjao @wadeking98 @esune

After authenticating with ACM test a keycloak error is displayed

After completing the proof request with ACM test a success banner is green briefly displays then the web transients to the Keycloak error noted in the screen shot below:

Steps to Reproduce

1. Goto ACM test
2. Authenticate with a test credential.

Follow up - Links alerts

The "alert details" link. The links do work, however you need to be logged into sysdig (an expected step), AND be viewing the correct sysdig team (unexpected step) to be directed to the alert trigger details. The links should automatically direct you to the correct sysdig team and alert notification.

Create a new ACA-Py Agent deployment for IDIM and plan for more

We would like to create a deployment of an ACA-Py agent for IDIM that will be an issuer for the BC Gov Verified Person credential. This should include the "normal" Dev / Test / Prod OpenShift workspaces, but the setup of IDIM will require some additional planning.

Task one is provision the environments and deploy an ACA-Py instance into the Dev workspace that we turn over to IDIM to use. They will provide the webhook URL, we'll given them a CANdy-Dev Endorser, and we'll provide API auth token. If you want we can move this task to another Issue.

The bigger picture is a bit more complicated. We think we have a plan, but wanted to go over it here and get @WadeBarnes input. Here are the needs:

• IDIM has 7 separate environments for the BC Services Card Mobile app and related services.
• Those environments are generally divided into Dev (3 environments), Test (3) and Prod (1).
• The general thinking is that we want 1 ACA-Py instance per IDIM environment right now as that will be the easiest way to manage the verifiable credential objects (DIDs, Schema, CredDefs, etc.).
  • Multi-tenant would be nice, but we're not there yet.

I don't think we want to have an OpenShift ACA-Py Dev/Test/Prod workspace set per IDIM instance -- that would be a lot of OpenShift environments. Rather, I'm thinking that we configure one OpenShift workspace set to have multiple instances of ACA-Py in each, one for each corresponding IDIM Instance. That would give us:

OpenShift Workspace	IDIM Environment	ACA-Py Name	Ledger
Dev	DEV	DEV	CANdy-Dev
Dev	DEV2	DEV2	CANdy-Dev
Dev	SIT	SIT	CANdy-Dev
Test	QA	QA	CANdy-Test
Test	Test	Test	CANdy-Test
Test	PreProd	PreProd	CANdy-Test
Prod	Prod	Prod	CANdy-Prod

For each instance, we would need pods for:

• ACA-Py
• Postgres

For at least one of the Test environments and Prod (and perhaps for all...)

• Redis
• PersistentQueues

@ianco

BC VC Pilot - change "Program" to be a drop down

Is this relatively easy to do? Ideally, this is already feature of Issuer Kit, but if not, we would want this to be a general feature of Issuer kit vs. specific to BC VC Pilot.

For now, use these values:

• LSBC
• DIT Testing
• IDIM Testing

We'll add more values as it makes sense.

update IDIM agent config for GA

The current connection name displayed in the BC Wallet for the IDIM agents is
IDIM (env), and should be changed to the more recognizable name BC Services Card

Change the default_label configuration property for all IDIM agents:
IDIM DEV default_label: BC Services Card (DEV)
IDIM SIT default_label: BC Services Card (SIT)
IDIM QA default_label: BC Services Card (QA)
IDIM PROD default_label: BC Services Card

This issue is also logged in IDIM JIRA as IASP-16231

Update Unverified Person Credential references

The test and "prod" OpenVP-CANdy issuer services have been migrated to their respective ledgers now that we have a full set of CANdy ledgers and not just dev.

The migration included creating new DIDs for the services. The old DIDs and any credentials issued by those DIDs are still valid, so references to the existing credentials should be retained.

Details of the new (and old) DIDs can be found here; bcgov/essential-services-delivery#124

Affects:

• https://github.com/bcgov/trust-over-ip-configurations/blob/main/proof-configurations/accredited-lawyer/test/accredited-lawyer-bcpc.json

Deploy Buy BC production agent/controller

We need an agent/controller for the production instance of Buy BC.

The agent will be unprivileged, and therefore some manual steps are required to endorse the creation of schema and cred_def.

Checklist:

• Deploy controller and agent in read-only mode (@esune)
• Register DID, schema, cred_def on ledger (@WadeBarnes)
• Restart agent in non-readonly mode (@esune / @WadeBarnes )
• Create connection between Buy BC agent and OrgBook agent (@esune / @WadeBarnes )

Stuck in an error when accessing PROD ACM app

I am getting a PROD lawyer credential + PROD person credential with a name mismatch. I was used to seeing a page showing that there is a name mismatch and the access would need to be approved. I am now getting a keycloak error page.

It's Been a While Since This Repository has Been Updated

This issue is a kind reminder that your repository has been inactive for 181 days. Some repositories are maintained in accordance with business requirements that infrequently change thus appearing inactive, and some repositories are inactive because they are unmaintained.

To help differentiate products that are unmaintained from products that do not require frequent maintenance, repomountie will open an issue whenever a repository has not been updated in 180 days.

• If this product is being actively maintained, please close this issue.
• If this repository isn't being actively maintained anymore, please archive this repository. Also, for bonus points, please add a dormant or retired life cycle badge.

Thank you for your help ensuring effective governance of our open-source ecosystem!

Deploy ACA-Py (0.7.4-rc3 nope) 0.7.4-rc4 for LSBC agent and fix the Revocation Registry issue

@WadeBarnes , @ianco -- with ACA-Py 0.7.4-rc3, we are ready to upgrade the prod deployment of the LSBC issuer to use the new release, and once deployed, to correct the RevReg problem that they are experiencing. Since this is a production problem going back more than 2 months, this is very high priority.

Please organize what has to happen for that and make that happen ASAP, so that all of the credentials that LSBC has revoked are seen as revoked. I think this means doing these steps:

• update dev to use new release and verify
  • who does the verification and what needs to get verified?
  • notify Wade that we can move to test
• update test to use the new release and verify
  • who does the verification and what needs to get verified?
  • notify Wade that we can go to prod
• update prod to use the new release and verify
  • What testing is done for this
  • Notify LSBC that the update has been done (do we normally do this? Do we normally notify them ahead of time?)
• Wade/Ian coordinate access to the issuer ACA-Py instance so that the new "fixRevReg" endpoint can be called
• Confirm that the ACA-Py state of the RevReg matches the ledger

Please let me know if I'm off base with the sequence, and what additional steps/people/notifications are needed to complete this.

We'd like to complete this is quickly as possible -- while being safe in going through the right steps.

Completing this by Monday morning would be really appreciated. I'm hoping that is possible, but do not know.

Add project lifecycle badge

No Project Lifecycle Badge found in your readme!

Hello! I scanned your readme and could not find a project lifecycle badge. A project lifecycle badge will provide contributors to your project as well as other stakeholders (platform services, executive) insight into the lifecycle of your repository.

What is a Project Lifecycle Badge?

It is a simple image that neatly describes your project's stage in its lifecycle. More information can be found in the project lifecycle badges documentation.

What do I need to do?

I suggest you make a PR into your README.md and add a project lifecycle badge near the top where it is easy for your users to pick it up :). Once it is merged feel free to close this issue. I will not open up a new one :)

Incorporate configurations for LCRB OrgBook issuer

Migrate and update the LCRB OrgBook issuer configurations from here.

DTS will be hosting/maintaining the issuer agent/controller for LCRB moving forward.

When done, create and submit PR to remove the component from the original repo as well.

Add table to readme to list all deployments in the repo and with a link to the config

Please add a table to the Readme that has an entry for every deployment in the repo. Entries should include:

• Name of deployment
• Purpose
• "Configuration" that is a link to the config data in the repo
• "Dev", "Test", "Prod" that are links to the deployment landing pages
• Notes

If you can at least put in the Name, purpose, configuration and live site fields, I can fill in the Purpose and Notes.

Thanks!

Create a cron job to check for LSBC revocation errors

It looks like it is going to be a couple more weeks before we get a fix for the LSBC issue about revocations. To deal with the issue over time, please do the following:

• Beginning immediately -- periodically, at least each weekday, run the fixRevReg (false) to see if there is a difference and if so, run the fixRevReg (true) endpoint to fix it. If someone else can run that, (perhaps @swcurran -- let us know how).
• Create a cron job that runs the fixRevReg (false) test every few minutes and sends a notification when a problem is detected.
• If you are comfortable fully automating the process, have the cron job also run the fixRevReg (true) step

Recognize that the need for this will only be here for a few more weeks, so balance out the effort for automating this (including testing / risk mitigation) with the time will need to do this.

@ianco is working on better fixes to eliminate the tracking concern.

uninstall_install_sandbox.yaml workflow runs on user forks

The uninstall_install_sandbox.yaml is not restricted to running on the main repository, so it is trying (and failing) to execute on user forks of the repository. For example, my fork; https://github.com/WadeBarnes/trust-over-ip-configurations/actions

This workflow is specific to the main repository and is dependent on the secrets available in the repo. The workflow should not be running on user forks.

BC VC Pilot -- invalid token allows use to create their own data

When a user tries to use an invalid token, they are allowed in and presented with an empty SurveyJS into which they can enter their own info and issue a credential to themselves. Hopefully that is just a configuration change needed, but if a user tries to use an invalid (or no) token on this site, they need to go to an error page -- perhaps the 404, or perhaps a page explicitly for this scenario.

This must be fixed, as we can't let people make their own credential.

@cvarjao @wadeking98 @esune

BCVCPilot is broken

It looks like there has been some recent update and the BC VC Pilot issuer is broken

• URLs are back being reusable (they should be one time use)
• The form is looks a bit weird (might just be me not remembering)

Please add ACAPY_INVITE_PUBLIC env variable to true to IDIM config (startup param --invite-public)

To test, please add it to, and redeploy:

https://github.com/bcgov/trust-over-ip-configurations/blob/main/openshift/templates/issuer-agent/issuer-agent-deploy.idim-sit.test.param

It's likely safe to add it to all of the IDIM controllers.

Remove PRIME, HG deployments

The Health gateway and PRIME agents are not being used, and should be decommissioned.

Complete the following steps:

• save a back-up of the agent wallets
• Remove entries from the backup configuration
  • postgres=wallet-prime/agent_prime_wallet
  • postgres=wallet-health-gateway/agent_health_gateway_wallet
• spin down the deployments (@WadeBarnes / @esune )
• remove configurations from repository/create tag for future reference (@WadeBarnes / @esune )
• remove configurations from OCP (@WadeBarnes / @esune / @rajpalc7 )
• close this issue (@WadeBarnes / @esune )

Incomplete webhook payloads for present-proof events

Recent testing from IDIM has raised an issue related to webhook payloads submitted for present-proof events: ACA-Py is not sending the full set of information contained in the presentation-exchange as expected, but rather a small subset of attributes without any details about the exchange. Querying the presentation-exchange record returns all of the information as expected.

{
  "connection_id": "0fcf9ad8-8237-40c3-a7e3-e3597c44239f",
  "role": "verifier",
  "initiator": "self",
  "auto_present": false,
  "auto_verify": false,
  "state": "verified",
  "thread_id": "ce430eba-d42c-4d00-8219-2dbd03249a3d",
  "trace": true,
  "verified": "true",
  "verified_msgs": [
    
  ],
  "created_at": "2024-01-24T00:21:43.075619Z",
  "updated_at": "2024-01-24T00:21:50.291356Z",
  "presentation_exchange_id": "9bb7d7e9-3308-46c9-9bb5-21d082c58cf1"
}

The last time this was tested in the IDIM environments was late 2022, so ACA-Py has been upgraded a few times since then however there haven't been changes in the presentation-exchange handlers in ACA-Py so this is a very odd issue/behaviour.

Deploy an new instance of Issuer Kit for BC VC Pilot credential

Please deploy a new instance of Issuer kit to issue a "BC VC Pilot" credential, with the following configuration. Once the main elements of the deployment is on Dev, we can revise the wording. The following are the parameters that I think you will need for deployment:

• Issue a credential called "BCVCPilot" schema with the attributes: "name"emailAddress", "program" and "iss_dateint"
• Nice to have: The credential should be revokable with a RevReg size of 1000 credentials.
  • We can add that after initial deployment, if necessary.
• Use ledger CANdy-Dev for writing.
• Enable the email invitation, with a database for entering the information for sending.
  • Require that a user have an active email validation code
  • So that all credential fields except "iss_datetime" about the Holder must be entered by the person Inviting the user.
• Don't include any other authentication to get the credential -- just the invitation code.
• In creating the credential, using the current date converted to an integer of the form "20220325" -- not a string.
• For the first cut, use the text from this instance of Issuer Kit "https://bcgov-citz-issuer.apps.silver.devops.gov.bc.ca/" for the invitation, landing page screen and post-issuance screen.
  • Please document where the text for this instance can be found so we can put in a PR to update it.
• Set the DNS for the landing page/site to "https://bcvcpilot.vonx.io"
• Set Clecio, Charles, John and myself as Admins for creating/sending/resending invitations on Dev, Test, and for Production - all except me.
• The flow should be:
  • Admin enters invitation/credential fields and sends email
  • Holder receives invitation and clicks link
  • Goes to landing page and clicks to continue
  • Since there is no other Auth except the invitation, goes to the "Enter Credential" form.
  • No data to enter, since all comes for the invitation data and the "Issue_dateint" is generated automagically
  • User clicks to continue
  • Confirmation screen - click to continue
  • QR code comes up, and then to issue flow
  • User taken to the post-issuance page
  • Done.

I think that's it. Let me know if any of these settings are difficult, or if I have left anything off.

Please ask @esune as needed for guidance.
@WadeBarnes heads up on this.
@cvarjao heads up on this

Would like this deployed as soon as is reasonable -- hoping for sometime next week.

Need to rotate the IDIM Revocation Registries after deploying ACA-Py 0.9.0

Two updates in ACA-Py 0.9.0 impact IDIM — some upgrades/fixes to the CL Signatures codebase, and the ability to rotate Revocation Registries. One of the fixes to CL Signatures addresses a bug that to fully address in a deployed environment requires rotating the RevRegs currently being used, so that all new issuances use the new RevRegs. To support rotating the RevRegs, a new endpoint has been added to the Admin API /revocation/active-registry/{cred_def_id}/rotate. For the IDIM instances, this needs to be run once after 0.9.0 has been successfully deployed and tested.

How to invoke the endpoint is an interesting question best handled by devs and devops folks. Easiest would be to run it as a one time event using the Swagger API or curl directly. Alternatively, some code could be put into the app to allow triggering when needed — although care would have to be taken to make sure that the process ran once and only once. There is a need to use this endpoint now, but is likely to never be needed again.

It's Been a While Since This Repository has Been Updated

This issue is a kind reminder that your repository has been inactive for 181 days. Some repositories are maintained in accordance with business requirements that infrequently change thus appearing inactive, and some repositories are inactive because they are unmaintained.

To help differentiate products that are unmaintained from products that do not require frequent maintenance, repomountie will open an issue whenever a repository has not been updated in 180 days.

• If this product is being actively maintained, please close this issue.
• If this repository isn't being actively maintained anymore, please archive this repository. Also, for bonus points, please add a dormant or retired life cycle badge.

Thank you for your help ensuring effective governance of our open-source ecosystem!

Tails Server Pipeline not working

When running the tails-server-pipeline, it fails with the following error:

OpenShift Build e79518-tools/tails-server-pipeline-31 from https://github.com/bcgov/trust-over-ip-configurations.git
Checking out git https://github.com/bcgov/trust-over-ip-configurations.git into /var/lib/jenkins/jobs/e79518-tools/jobs/e79518-tools-tails-server-pipeline/workspace@script/fa51c42de219a7a3deaf03008528c7ef91f93a7d5c60f093f6b2fa1f40128220 to read jenkins/tails-server/Jenkinsfile
The recommended git tool is: NONE
No credentials specified
 > git rev-parse --resolve-git-dir /var/lib/jenkins/jobs/e79518-tools/jobs/e79518-tools-tails-server-pipeline/workspace@script/fa51c42de219a7a3deaf03008528c7ef91f93a7d5c60f093f6b2fa1f40128220/.git # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/bcgov/trust-over-ip-configurations.git # timeout=10
Fetching upstream changes from https://github.com/bcgov/trust-over-ip-configurations.git
 > git --version # timeout=10
 > git --version # 'git version 2.31.1'
 > git fetch --tags --force --progress -- https://github.com/bcgov/trust-over-ip-configurations.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git rev-parse origin/main^{commit} # timeout=10
Checking out Revision 430416d23fcce0178821190dc2afc6656b2f2e96 (origin/main)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 430416d23fcce0178821190dc2afc6656b2f2e96 # timeout=10
Commit message: "Merge pull request #140 from WadeBarnes/main"
 > git rev-list --no-walk 430416d23fcce0178821190dc2afc6656b2f2e96 # timeout=10
[Pipeline] Start of Pipeline
[Pipeline] library
WARNING: Unknown parameter(s) found for class type 'jenkins.plugins.git.GitSCMSource': branches
Loading library custom-lib@main
Attempting to resolve main from remote references...
 > git --version # timeout=10
 > git --version # 'git version 2.31.1'
 > git ls-remote -- https://github.com/bcgov/trust-over-ip-configurations.git # timeout=10
Found match: refs/heads/main revision 430416d23fcce0178821190dc2afc6656b2f2e96
The recommended git tool is: NONE
No credentials specified
 > git rev-parse --resolve-git-dir /var/lib/jenkins/jobs/e79518-tools/jobs/e79518-tools-tails-server-pipeline/workspace@libs/4bd37be607a9bd1783e30b46421c8552d8e2ee9a6d09d5cc7ec374419d683eeb/.git # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/bcgov/trust-over-ip-configurations.git # timeout=10
Fetching without tags
Fetching upstream changes from https://github.com/bcgov/trust-over-ip-configurations.git
 > git --version # timeout=10
 > git --version # 'git version 2.31.1'
 > git fetch --no-tags --force --progress -- https://github.com/bcgov/trust-over-ip-configurations.git +refs/heads/*:refs/remotes/origin/* # timeout=10
Checking out Revision 430416d23fcce0178821190dc2afc6656b2f2e96 (main)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 430416d23fcce0178821190dc2afc6656b2f2e96 # timeout=10
Commit message: "Merge pull request #140 from WadeBarnes/main"
[Pipeline] node
Running on [Jenkins](https://jenkins-e79518-tools.apps.silver.devops.gov.bc.ca/computer/(built-in)/) in /var/lib/jenkins/jobs/e79518-tools/jobs/e79518-tools-tails-server-pipeline/workspace
[Pipeline] {
[Pipeline] load
[Pipeline] // load
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
java.nio.file.NoSuchFileException: /var/lib/jenkins/jobs/e79518-tools/jobs/e79518-tools-tails-server-pipeline/workspace@script/jenkins/tails-server/config.groovy
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
	at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219)
	at java.base/java.nio.file.Files.newByteChannel(Files.java:371)
	at java.base/java.nio.file.Files.newByteChannel(Files.java:422)
	at java.base/java.nio.file.Files.readAllBytes(Files.java:3206)
	at java.base/java.nio.file.Files.readString(Files.java:3284)
	at hudson.FilePath$ReadToString.invoke(FilePath.java:2377)
	at hudson.FilePath$ReadToString.invoke(FilePath.java:2372)
	at hudson.FilePath.act(FilePath.java:1192)
	at hudson.FilePath.act(FilePath.java:1175)
	at hudson.FilePath.readToString(FilePath.java:2369)
	at org.jenkinsci.plugins.workflow.cps.steps.LoadStepExecution.start(LoadStepExecution.java:39)
	at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:322)
	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:196)
	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:124)
	at jdk.internal.reflect.GeneratedMethodAccessor445.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1225)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:41)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:180)
	at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:162)
	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:178)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:182)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:152)
	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
	at WorkflowScript.run(WorkflowScript:13)
	at ___cps.transform___(Native Method)
	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:90)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:116)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:85)
	at jdk.internal.reflect.GeneratedMethodAccessor247.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:110)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:85)
	at jdk.internal.reflect.GeneratedMethodAccessor247.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.CastBlock$ContinuationImpl.cast(CastBlock.java:44)
	at jdk.internal.reflect.GeneratedMethodAccessor257.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.CollectionLiteralBlock$ContinuationImpl.dispatch(CollectionLiteralBlock.java:55)
	at com.cloudbees.groovy.cps.impl.CollectionLiteralBlock$ContinuationImpl.item(CollectionLiteralBlock.java:45)
	at jdk.internal.reflect.GeneratedMethodAccessor258.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
	at com.cloudbees.groovy.cps.Next.step(Next.java:83)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:158)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:152)
	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:136)
	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:275)
	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:152)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:187)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:420)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:95)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:330)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:294)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:30)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:70)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Finished: FAILURE

See https://jenkins-e79518-tools.apps.silver.devops.gov.bc.ca/job/e79518-tools/job/e79518-tools-tails-server-pipeline/30/console for the actual pipeline run.

BC VC Pilot -- remove the double email address?

Is there a way to remove the double entry of the email address in the Admin form? Perhaps (yet another) a config setting that named the claim into which the email address should go?

Issuer agent instances being heavily throttled in Demo Apps (a99fd4) environments

Several of the issuer agent instances are being throttled at >50% on average. Review and adjust the CPU resource allocations, primarily the CPU limit to reduce or eliminate the throttling. The goal should be to reduce throttling to <25% on average. For production an even lower average may be desirable.

These metrics can be easily reviewed using the Namespace Monitoring dashboard available through Grafana in our new monitoring stack.

Affected instances:

• BC VC Pilot
• A2A

Follow up - Links alert

Define the text on the BC VC Pilot

@alexgmetcalf and Kim -- there are several places in the BC VC Pilot issuer that we can adjust the wording of the text visible to people. Can you please take the words and provide new, improved, 100% better words?

I'm sending you an email invitation to the service so that you can try it out and see where the words to be changed come into play.

The specific chunks of text needed are:

• The email invitation, also found here.
• The landing/acknowledgment page after clicking the link in the email, text also found here.
• The "after issue credential" landing page (couldn't find it in the source code).

I suggest that you run through the process and as you do, copy the email text, landing page text, and post-issue page text and paste each into a google doc. Then revise the text in the Google Doc and add a comment to this issue that includes the link to the Google Doc. We (@wadeking98 or I) will do a pull request to the repo to apply your changes.

Let us know if you have any questions.