3keycompany/czertainly-interfaces

CZERTAINLY - interfaces definitions and common objects for the platform

czertainly-interfaces's Issues

Failed validation of attributes should send back the attribute label instead of name

The Core-Interfaces provides the validation methods for attributes:
https://github.com/3KeyCompany/CZERTAINLY-Interfaces/blob/develop/src/main/java/com/czertainly/core/util/AttributeDefinitionUtils.java

When the attribute is wrong, for example it is required, but missing, the Exception is sent back with the name of the attribute that is wrong:

errors.add(ValidationError.create("Required attribute {} not found.", definition.getName()));

To improve the user behaviour, we should send back the name and label, because the users operating on the web interfaces see the labels of the attributes only.

Add CodeBlock attribute content type

Add new attribute content type for storing code snippets.

Requirements:

• its data should contain information which language is code written in
• code text should be BASE64 encoded to keep formatting intact including whitespaces

Define API routes for managing custom attributes

Define new API routes used to manage custom attributes.

Requirements:

• routes for CRUD operations for custom attributes and global metadata
• routes to enable/disable custom attributes
• route to assign resources where custom attribute is applicable
• include custom attributes in response DTOs for retrieving object detail

Secret attribute content in Credential attribute is masked also when sending to connector

Credential attribute content data is CredentialDto which contains secret attribute content as ResponseAttributeDto. Since we introduced custom serializer for ResponseAttributeDto, it is masked in Credential attribute content even when sent to connector which results in null pointer when using secret in credential.

CredentialAttributeContentData needs to be created which will not contain ResponseAttributeDto to prevent masking.

Add identify certificate endpoint to Authority provider interface

Add endpoint used to identify certificate by authority by the sent certificate content. It is used to identify certificate that is migrated to the authority to retrieve attributes/metadata necessary to perform certificate operations.

Response:

• 404 - certificate is unknown to authority or does not have necessary information about certificate
• 200 - certificate is identified and certificate attributes are sent in response

Add endpoint to issue existing certificate in status New

New endpoint should be added to Client operations v2 and contain no body and its response should be same as existing issue endpoint.

• add endpoint with route path: /v2/operations/authorities/{authorityUuid}/raProfiles/{raProfileUuid}/certificates/{certificateUuid}/revoke
• since this new endpoint will replace same functionality that is in existing issueCertificate endpoint, remove uuid property from ClientCertificateSignRequestDto.

Update search fields DTOs to allow search on custom and meta attributes

Update search fields DTOs to allow searching not only on properties of resource but also on custom and metadata attributes.

• add boolean type to SearchableFieldType with EQUALS condition
• getSearchableFieldInformation endpoint should return also list of available custom attributes and metadata to search on
• SearchFilterRequestDto needs to have search field source type to properly identify field (property, custom or meta attribute)
  add available custom and meta attributes for resource to response of filter searchable fields

Extend Authority Provider interface to get CA certificates and CRL

We should be able to get CA certificate, or full chain of CA certificates up to root, from the RA Profile, including the current CRL if available and issued by the authority.

• endpoint for retrieving CA certificates should be mandatory and should take in account certificate type
• endpoints for CRL should be optional

Improper validation of BaseAttributeContent

com.czertainly.core.util.AttributeDefinitionUtils
In the validation of attributes there is a section with common cases:

case STRING:
                    case INTEGER:
                    case SECRET:
                    case BOOLEAN:
                    case FLOAT:
                    case TEXT:
                        BaseAttributeContent<?> stringBaseAttributeContent = (BaseAttributeContent)ATTRIBUTES_OBJECT_MAPPER.convertValue(baseAttributeContent, BaseAttributeContent.class);
                        if (stringBaseAttributeContent.getValue() == null) {
                            errors.add(ValidationError.create("Wrong value of Attribute {} {}.", new Object[]{definition.getName(), definition.getType()}));
                            wrongValue = true;
                        }
                        break;

It does not check for the proper types of the values inside the content.
For example, when the attribute is defined as INTEGER and client will send string in the value, for example "value": "5986", it will not be evaluated as wrong attribute, but it should.

We should include in this section validation of the value with the attribute type definition.

Define Cryptographic provider connector interface

Define endpoints for cryptographicProvider function group connectors.

• add controller/s with endpoints required to be implemented by cryptographic provider connector
• define necessary DTOs

Add certificate chain download endpoint

Add new endpoint GET /certificates/{uuid}/chain/{certificateFormat} to get BASE64 encoded certificate chain in specified format.
Path parameter certificateFormat represents new enum type CertificateFormat with two supported enum items PKCS7 and PEM. In addition, add optional boolean query parameter withEndCertificate.

Response of chain download is CertificateChainDownloadResponseDto with following properties

• completeChain - boolean flag if full certificate chain could be constructed
• format - same enum as in request to specify what format is in output
• content - BASE64 encoded certificate chain in specified format

Accordingly change response of get certificate chain endpoint to CertificateChainResponseDto

• completeChain - boolean flag if full certificate chain could be constructed
• certificates - list of certificates as CertificateDetailDto

Requirements:

• add new endpoint to download certificate chain in specified format, with options specified by query params
• add support for same query param also to endpoint GET /certificates/{uuid}/chain
• add new enum CertificateFormat implementing IPlatformEnum with 2 items
  • code: pkcs7, label: PKCS#7
  • code: pem, label: PEM

Add SCEP URL to listing DTO

Move SCEP URL property from detail DTO to listing DTO.

Consolidate POM dependencies between project with help of parent POM file

Right now, dependencies between projects are in various different versions, even somewhere without version which then uses latest version which is not desirable.

It is necessary to consolidate and cleanup dependencies and create POM hierarchy with defined parent POM file with desired dependencies versions across platform.

Add support for Cryptography related items in the interface

Add the interfaces for the management of the following items.

1. Tokens
2. Token Profile
3. Cryptographic Keys
4. Key Operations

Add the DTOs and the needed updates to the existing DTOs for the same

Define interface for management of approval profiles

Define core interface to manage approval profiles with DTOs corresponding to DB entity model defined in https://github.com/orgs/3KeyCompany/projects/8/views/1?pane=issue&itemId=30534377

Route mapping: /v1/approvalProfiles

Approval profile should have following operations

• list (DTO has properties of ApprovalProfile + number of steps of approval profile)
• detail - returns approval profile with its steps
• create/edit approval profile with its steps
• enable/disable operation
• delete operation

Define interface for management of approvals

Define core interface to manage approvals with DTOs corresponding to DB entity model defined in https://github.com/orgs/3KeyCompany/projects/8/views/1?pane=issue&itemId=30534377

Route mapping: /v1/approvals

Approvals should have following operations

• listing
• detail - returns approval with current status of its steps and list of users that interacted with it in the process
• approve whole approval - PATCH /v1/approvals/{uuid}/approve
• reject whole approval - PATCH /v1/approvals/{uuid}/reject
• approve by single recipient - PATCH /v1/approvals/{uuid}/approveRecipient (needs DTO with comment)
• reject by single recipient - PATCH /v1/approvals/{uuid}/rejectRecipient (needs DTO with comment)

Add new endpoint to certificate controller to list certificate's approvals

• /v1/certificates/{certificateUuid}/approvals

Update group and users DTO to allow assigning users to group

• add email to Group DTOs
• add group UUID and group name to user DTOs for communicating with Auth service (package com.czertainly.api.model.core.auth)
• add group UUID to DTOs to communicating with Core for create and update user (package com.czertainly.api.model.client.auth)

Change certificate detail DTO to contain reference to certificate request

Add new property certificateRequest to certificate detail DTO which contains CertificateRequestDTO representing certificate request.

DTO should have following properties:

• certificate type
• format
• algorithm
• signature algorithm
• content
• common name
• subject DN
• attributes
• signature attributes
• SANs

Add endpoint to retrieve certificates of authority belonging to RA profile

Add new endpoint to RA profile management controller to retrieve authority certificate chain.

GET /v1/authorities/{authorityUuid}/raProfiles/{raProfileUuid}/caCertificates with response type as list of <CertificateDetailDto>

Update custom attributes listing DTO to contain list of resources

Create separate DTO for listing custom attributes that in addition contains list of resources for each custom attribute. It can be moved from detail DTO since it is extending listing one.

Define endpoints to manage cryptographic tokens and token profiles to use functionality of cryptographic provider

Add list of related certificates to certificate detail DTO

Add related certificates to CertificateDetailDto that will be populated with CertificateDto representing certificates for which certificate is source.

Implement interface to communicate with Scheduler service

Implement interface for communication with Scheduler service to manage scheduled jobs.

• Design and implement DTOs used in API endpoints of Scheduler service
• implement API client for Scheduler service

Improve masking of Secrets attribute content

Content of Secret attribute content type needs to be automatically masked before sending it in response. It will prevent that secrets are leaked when explicit masking function is not applied.

Add endpoints for assigning approval profile to RA profile

Add endpoints that allows to get and associate/dissasociate existing approval profile to RA profile

• GET /v1/authorities/{authorityUuid}/raProfiles/{raProfileUuid}/approvalProfiles
• PATCH /v1/authorities/{authorityUuid}/raProfiles/{raProfileUuid}/approvalProfiles/{approvalProfileUuid}
• DELETE /v1/authorities/{authorityUuid}/raProfiles/{raProfileUuid}/approvalProfiles/{approvalProfileUuid}

Add bulk operations to internal notifications interface to delete and mark as read multiple notifications

Add two endpoint for bulk operation with body as list of UUIDs to perform operation on.

• PATCH /v1/notifications - to bulk mark as read
• DELETE /v1/notifications - to bulk delete

Add endpoint to retrieve CA certificate chain for RA profile

Add endpoint to RA Profile Management interface to get the issuing CA certificate chain from connector.

GET /v1/{authorityUuid}/raProfiles/{raProfileUuid}/caCertificates with response - list of certificate detail DTOs.

Define API routes for managing global metadata

Define new API routes used to manage global metadata.

Requirements:

• CRUD operations for custom attributes and global metadata

Extend discovery Core interface to handle external discoveries

Add possibility to run discoveries manually in external provider. To achieve that, discovery Core interface and DTOs representing discovery needs to be extended:

• discovery can be created as external (bool)
• add endpoint to get discovery configuration - GET DiscoveryRequestDto
• add endpoint to import external discovery result - POST DiscoveryProviderDto
• add endpoint to import externally discovered certificates - POST List<DiscoveryProviderCertificateDataDto>

Define general interface and structure for platform enums

Right now, enums of platform are implemented differently and does not contain same fields that represent their items consistently compared with each other.

Each enum item should contain following properties:

• code - value used in communication that uniquely identifies enum item and should not contain special characters that could complicate its storage, formatting and usage in JSON formatting
• label - string representing item and used as display name for item (e.g. in selection, messages, etc.)
• description - optional, longer description of meaning and value of item (e.g. possible usage as tooltip)

Endpoints for retrieving enums and their items should be defined. That way, enums can be used not only by code, but also enhanced with other properties as name and description.

Example:

GET /enums
{
    "AttributeType": {
        "data": {
            "code": "data",
            "label": "Data"
        },
        "meta": {
            "code": "meta",
            "label": "Metadata"
        },
        "custom": {
            "code": "custom",
            "label": "Custom"
        },
        "group": {
            "code": "group",
            "label": "Group"
        },
        "info": {
            "code": "info",
            "label": "Info"
        }
    },
    "KeyType": {
        "private": {
            "code": "private",
            "label": "Private key",
            "description": "Asymmetric private key"
        },
        "split": {
            "code": "split",
            "label": "Split key",
            "description": "Secret or private key split into parts"
        },
        "public": {
            "code": "public",
            "label": "Public key",
            "description": "Asymmetric public key"
        },
        "secret": {
            "code": "secret",
            "label": "Secret key",
            "description": "Symmetric secret key"
        }
    }
}

Define endpoint for retrieving internal notification messages

Add new controller for notifications with endpoint to retrieve user notifications with query parameter serving as filter for unread messages. If query parameter unread is not present retrieve all messages, otherwise based on boolean value.

v1/notifications?unread=true

Add endpoints to mark as read and delete notification for specific user

Define and implement interface for compliance provider function group

1. Create End Points Controller interface for Compliance Connector related end-points

• Create the interface for the end points needed for Compliance Provider. The details of the end points can be found at https://wiki.3key.company/en/3Key_Services/RA_Profiles/Architecture/Compliance_Profiles/Connector

2. Create DTO Objects for Compliance Provider

• Create the DTO Objects that can be used the Compliance Providers. These objects should include the commonly used classes like input and output for the Compliance API

3. Create End Points Controller interface for Compliance Profile for Core related items

• Create the interface for end points that are needed by the core to implement the compliance profiles. These end points should include, Compliance Profile items, Compliance Validation End Points for the Certificate and Compliance Profile tagging APIs. It should comply with the design provided at https://wiki.3key.company/en/3Key_Services/RA_Profiles/Architecture/Compliance_Profiles/Core

4. Create DTO Objects for Core items

• Create the DTO objects that should include
  • Client DTO used by the clients to send request to the Core
  • Core DTOs used by the Core to send response back to the clients

Separate DTOs for Discovery listing and detail operation

Discovery Listing and Discovery Details API are currently using DiscoveryHistoryDto. This DTO contains the complete information that is only needed for the details page.

Much of the costly information in the data is not being used by the List page in the FE and is also not relevant.

• DiscoveryHistoryDTO should be renamed to DiscoveryHistoryDetailDto
• DicoveryHistoryDetailsDto should be used only by discovery details API
• Create a new DiscoveryHistoryDTO that contains UUID, name, total certificates, status, connector and kind
• Discovery list API should return a list of newly created DiscoveryHistoryDTO

Define structure of rules applicable to discovered certificates

After certificates are found by discovery, rules assigned to discovery should be applied to each certificate. Rules consist of defined criteria on properties of certificate and action that needs to be performed when certificate matches criteria.

Define DTOs and interface to manage discovery rules.

Bug in the BaseApiClient CERTIFICATE authentication

In https://github.com/3KeyCompany/CZERTAINLY-Interfaces/blob/develop/src/main/java/com/czertainly/api/clients/BaseApiClient.java, there is a bug in the SslContext:

TrustManager tm = null;
            String trustStoreData = AttributeDefinitionUtils.getAttributeValue(ATTRIBUTE_TRUSTSTORE, attributes);
            if (trustStoreData != null && !trustStoreData.isEmpty()) {
                TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm()); //"SunX509"

                String trustStoreType = AttributeDefinitionUtils.getAttributeValue(ATTRIBUTE_TRUSTSTORE_TYPE, attributes);
                String trustStorePassword = AttributeDefinitionUtils.getAttributeValue(ATTRIBUTE_TRUSTSTORE_PASSWORD, attributes);
                byte[] trustStoreBytes = Base64.getDecoder().decode(keyStoreData);

                tmf.init(KeyStoreUtils.bytes2KeyStore(trustStoreBytes, trustStorePassword, trustStoreType));
                tm = tmf.getTrustManagers()[0];
            }

byte[] trustStoreBytes = Base64.getDecoder().decode(keyStoreData); should decode trustStoreData.

            return SslContextBuilder
                    .forClient()
                    .keyManager(km)
                    .trustManager(tm)
                    .protocols("TLSv1.2")
                    .build();

When the tm is null, it will throw NullPointerException.

Fix validation of custom attributes

Validation of custom attributes is currently skipped because utils method implemented in Interfaces takes only data attributes definitions into account and filters out other attributes types.

Validation should support also custom attributes and check same properties and content to not allow create object without required custom attributes.

Validation of LIST attributes is not implemented

The validation of LIST attributes is commented out in the:
https://github.com/3KeyCompany/CZERTAINLY-Interfaces/blob/develop/src/main/java/com/czertainly/core/util/AttributeDefinitionUtils.java

            case LIST:
//                wrongValue = !((Collection) definition.getValue()).contains(attribute.getValue());
                break;

There is no reason described in the code, why it is commented. The validation of LIST attribute is therefore not working, when wrong values (or incompatible values) are sent as part of the request.

This is connected with the 3KeyCompany/CZERTAINLY-FE-Administrator#37

The validation of the LIST attribute should be implemented, checking its values, multivalues if defined, and also the callback values if available.

Update DTOs with UUIDs of related objects

Some DTOs representing listed objects are missing UUIDs of related objects, which prevents to construct request if user would like to retrieve detail of it.

Make following changes to DTOs:

• add ownerUuid to KeyDto and KeyDetailDto
• remove owner in KeyRequestDto
• replace owner with ownerUuid in EditKeyRequestDto
• use SimplifiedRaProfileDto when representing RA profile in
  • AcmeAccountListResponseDto
  • AcmeAccountResponseDto
  • AcmeProfileListDto
  • AcmeProfileDto (replace current RaProfileDto)
  • ScepProfileDto
  • ScepProfileDetailDto (replace current RaProfileDto)

Add endpoints for managing settings of platform

Add endpoints to get and update settings for CZERTAINLY platform that can be edited by user with access

Endpoints:

GET /v1/settings/sections -> list of sections = enum items
GET /v1/settings/sections/{sectionName} -> section settings DTO
PUT /v1/settings/sections/{sectionName} -> update section settings

Requirements:

• settings should belong to sections (enum)
• define DTOs for individual setting options

Add endpoints for management of notification instances

Individual notifications can be configured and used in form of notification instance references that will be managed by Core.
Similarly as authority instance references, notification instance references hold reference to its counterpart stored in provider.

• based on the notification provider interface, define endpoints used by Core clients to manage and use notification instances.
• implement API client that can be used to communicate with notification provider connectors
• define DTO representing mapping attributes required by provider

NotificationInstanceDTO

• uuid
• name
• description
• connectorUuid
• connectorName
• kind
• attributes (connector attributes definitions from connector)
• mapped attributes (saved mapping from mapping attributes of provider to custom attributes definitions)

NotificationInstanceRequestDTO

• name (just in create)
• description
• connectorUuid (just in create)
• kind (just in create)
• attributes (RequestAttributeDto)
• mapped attributes (mapping from mapping attributes of provider to custom attributes definitions)

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Update actions/checkout digest to a5ac7e5
• Update dependency org.apache.maven.plugins:maven-gpg-plugin to v3.2.4

Detected dependencies

github-actions

.github/workflows/build.yml

• actions/checkout v4@b4ffde65f46336ab88eb53be808477a3936bae11
• actions/setup-java v4
• actions/cache v4
• actions/checkout v4@b4ffde65f46336ab88eb53be808477a3936bae11
• actions/setup-java v4
• actions/cache v4
• actions/cache v4
• actions/github-script v7

.github/workflows/codeql.yml

• actions/checkout v4@b4ffde65f46336ab88eb53be808477a3936bae11
• github/codeql-action v3
• actions/setup-java v4
• actions/cache v4
• github/codeql-action v3

.github/workflows/publish-package.yml

• actions/checkout v4@b4ffde65f46336ab88eb53be808477a3936bae11
• actions/setup-java v4
• actions/setup-java v4

.github/workflows/workflow_run_pruner.yml

• actions/github-script v7
• actions/github-script v7

maven

pom.xml

• com.czertainly:dependencies 1.1.0
• org.apache.maven.plugins:maven-gpg-plugin 3.1.0

---

• Check this box to trigger a request for Renovate to run again on this repository

Fix CertificateInLocationDto attribute properties type

Define API routes for managing collections

Define new API routes used to manage collections.

Requirements:

• CRUD operations for collections
• enhance interfaces attribute utils with validation of attributes with collection content

Fix update RA profile from the ACME profile description

https://docs.czertainly.com/api/core-acme/#operation/updateRaProfile
it is delete, not update

Add endpoint to retrieve certificate chain

• add endpoint to CertificateController to retrieve certificate chain GET /v1/certificates/{uuid}/chain with response of List<CertificateDto>
• add issuerCertificateUuid property to CertificateDto (not required)

Define Entity provider interface

Define new function group for entity provider and define its interface.

The Entity Provider implements the following mandatory interfaces:

• Info API
• Health check API
• Attributes API
• Entity management API
• Certificate locations API
• Certificate management API

Define interface for Notification provider function group

Add web interface for management of scheduled jobs

Add interface for management of scheduled jobs to Core API. It should contain following endpoints

• list scheduled jobs GET /v1/scheduler/jobs
• detail of scheduled job GET /v1/scheduler/jobs/{jobUuid}
• detail of scheduled job DELETE /v1/scheduler/jobs/{jobUuid}
• get history of scheduled job history GET /v1/scheduler/jobs/{jobUuid}/history
• enable scheduled job PATCH /v1/scheduler/jobs/{jobUuid}/enable
• disable scheduled job PATCH /v1/scheduler/jobs/{jobUuid}/disable

Scheduled job has following properties

• uuid
• name
• scheduling settings (cron expression)
• job type - job processor identification
• system - flag if scheduled job is managed by CZERTAINLY)
• enabled - is enabled/disabled
• repeated - boolean flag if job is one time triggered or repeated
• blocking - boolean flag if to allow more triggers of same job running in parallel
• job parameter data

Scheduled job history has following properties

• start time
• end time
• job ID - internal ID of scheduler microservice
• status
• result data (message, response)

Document how to get fieldIdentifier in case of searching for objects

The API for searching, for example listCertificates has field with name fieldIdentifier.

It is not clear how to get these values to be filled for the request and improving the documentation of the API should help users to know how to get supported values from https://docs.czertainly.com/api/core-certificate/#tag/Certificate-Inventory/operation/getSearchableFieldInformation.

Add settings section for notifications

• add new enum item to SettingsSection with code 'notifications'
• add endpoints for saving and retrieving notifications settings
  • GET /v1/settings/notifications
  • PUT /v1/settings/notifications
• DTO used for both endpoint should have one property notificationsMapping represented by type Map<NotificationType, UUID>