epam/edp-headlamp

The single management point of all the EDP resources

EDP Portal

Central management tool in the EDP ecosystem. Powered by Headlamp.

❗ Please refer to EDP documentation to get the notion of the main concepts and guidelines.

Overview

EDP Portal is built on the top of Headlamp. All EDP-specific functionality is written as Headlamp plugins.

overview.mp4

Assets

Installation

In order to install the EDP Headlamp, follow the steps below:

1. To add the Helm EPAMEDP Charts for local client, run "helm repo add":

   helm repo add epamedp https://epam.github.io/edp-helm-charts/stable

2. Choose available Helm chart version:

   helm search repo epamedp/edp-headlamp -l

   Example response:

   NAME                	            CHART VERSION	APP VERSION	    DESCRIPTION
   epamedp/edp-headlamp	            0.13.0        	0.13.0      	A Helm chart for EDP Headlamp

   NOTE: It is highly recommended to use the latest released version.

3. Full chart parameters available in deploy-templates/README.md.

4. Install edp-headlamp in the namespace with the helm command; find below the installation command example:

   helm install edp-headlamp epamedp/edp-headlamp --namespace <edp-project> --version <chart_version> --set name=edp-headlamp --set global.platform=<platform_type>

5. Check the namespace that should contain edp-headlamp deployment in a running status.

Local Development

Development versions are also available, please refer to the snapshot Helm Chart repository page.

Requirements

• NodeJS LTS v16.16.0 or higher.
• Headlamp Desktop App v0.10 or higher.
• KUBECONFIG environment variable is configured.

Start Local Development

• Run the npm run start command.
• Open the Headlamp desktop application.

edp-headlamp's Issues

Headlamp should contain configuration section for DefectDojo integration

In headlamp we have section (like gitserver, remote cluster) which contains credentials for DefectDojo integration

Acceptance Criteria:

• user can manage DD secret with Configuration section;
• inherit behaviour as for secrets for registry/sonar/nexus integration;
• in EDP env can be create only ONE secret;
• user cannot redefine name "defectdojo-ciuser-token";
• add description for each fields in DefectDojo section;
• in case if secret have ownerRef (managed by external-secret operator) lets hide button to modify object (save/remove) and add label managed-by - External secret operator (get value from secret object) like default branch

Secret Structure:

apiVersion: v1
kind: Secret
metadata:
  name: defectdojo-ciuser-token
  namespace: <edp-project>
  labels:
   app.edp.epam.com/secret-type: defectdojo
data:
  token: <base64_token>
  url: <base64_url>
type: Opaque

Connect headlamp's filter logic with marketplace page, add pagination and status handling

At the moment market place has no filter logic(search/namespace). Also, it doesn't have pagination and error/loading status handling. Need to implement those.

Provide link to QG pipeline in CD part

As EDP User, I want to have valid link from Headlamp UI to QG pipeline in CD part.

Acceptance Criteria:

• QG pipeline have valid link to Tekton pipeline;

As an EDP User, I want to use custom certificate in Headlamp deploy

https://epam.github.io/edp-install/operator-guide/manage-custom-certificate/?h=cert

Acceptance Criteria:

• EDP user can use own certificate to deploy Headlamp;
• add headlamp component to mkdocs article;
• use the same approach as for other components, described on mkdocs article;
• epam/edp-nexus-operator@5fd06c9

Add notifications on deploy/update/uninstall buttons click in applications section on stage page

Add redirect from marketplace to application

Add notification about unset namespaces

Acceptance Criteria:

• in case if EDP user use cd pipelines we need to enable new namespaces to allow list in Headlamp settings;
• add notification and link for user to define default and allowed ns;

Remove edpName usage

Migrate from edpName usage to ReleaseName as default variable in helm chart.
https://github.com/epam/edp-install/blob/master/deploy-templates/values.yaml#L10

AC:

• align remove edpName from values.yaml
• align helm Release Name usege;
• align documentation;

Forced branch version postfix (RC) on a Create new release form prevents to create release branches

Describe the bug

The Create new branch form forces to fill the branchVersionPostfix field for release branches.
In previous EDP versions the use of the postfix was not enforced.

EDP Versions

EDP: v3.3.0
Tekton: v0.45.0
Headlamp image tag: 0.6.0

Screenshots

EDP v3.3.0 with Headlamp:

EDP v2.12.1 with Admin Console:

Move Up the Create App button on the Overview Headlamp page

The button to open the dialog for adding a EDP component at the bottom of the page and it is not always convenient and visual, let's raise it up.

Acceptance Criteria:

• button to open the dialog for onboarding EDP applications can be easily found on the top of Overview Headlamp page;

Make grid mode for overview list page

Make grid mode for overview list page

Editor doesn't work

Provide link to Kibana logs from headlamp for the deployed applications

Acceptance Criteria:

• EDP user can open kibana dashboards which point to deployed Stage
• EDP user can open kibana dashboard for the specific application
• check for edpcomponents https://github.com/epam/edp-install/blob/master/deploy-templates/values.yaml#L289
• ensure documentation updated
• Use icon https://icon-sets.iconify.design/material-symbols/find-in-page-outline-rounded/ and put it near ArgoCD icon, caption: "Inspect logs", add symbol meaning that link will be opened in new window
• For stage level open kibana with the link, where NAMESPACE can be obtained from stage.spec.namespace or you can construct it, for the following link NAMSPACE=edp-delivery-sk-dev-search-feat-dev:
• https://kibana.example.com/app/discover#/?_g=()&_a=(columns:!(),filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,key:kubernetes_namespace_name,negate:!f,params:(query:edp-delivery-sk-dev-search-feat-dev),type:phrase),query:(match_phrase:(kubernetes_namespace_name:edp-delivery-sk-dev-search-feat-dev)))),interval:auto,query:(language:kuery,query:''),sort:!(!('@timestamp',desc)))

Headlamp should contain configuration section for registry integration

In headlamp we have section (like gitserver, remote cluster) which contains credentials for harbor integration

Acceptance Criteria:

• User is able to create secret for harbor integration
• Secret has agreed label
• User is able to manage secret

Secret Structure:

kind: Secret
metadata:
  name: kaniko-docker-config
  namespace: <edp-project>
type: Opaque
stringData:
  config.json: |
    {
      "auths" : {
        "harbor-registry.com": { "auth": "<base64 encoded "user:secret" string>" }
      }
    }

Where:

• kaniko-docker-config - Secret Name;
• harbor-registry.com - Registry endpoint;
• User - User for integration with Registry;
• Secret - Secret for integration with Registry;
• to generate in base64 use command - echo -n "user:secret" | base64;
• example - echo -n "robot$edp-project+edp:secret" | base64;
• use label to display and create secret: app.edp.epam.com/secret-type=registry;
• provide example for values to make sure user know pattern for data;

Update links on headlamp side

Align secrets fields in Platform UI configuration

As an EDP user I want to have opportunity integrate Headlamp with OKD oauth server

Acceptance Criteria:

• add feature to enable Headlamp integration with OKD oauth server;

Fix image tags real-time updating

STR:

Create any application and test codebases
Run build pipeline for application
Create CD Pipeline with QG autotest
Deploy the application and promote using autotest
After promotion go to the applications tab and open Image stream version popup 

AR: Stable version isn't displayed in the Image stream version popup without the page refreshing

ER: Stable version should be displayed in the Image stream version popup

Align button size, remove button name

Provide link to Grafana from Headlamp for the deployed applications

Acceptance Criteria:

• EDP user can open grafana dashboards which point to deployed Stage
• EDP user can open grafana dashboard for specific application
• align edpcomponents https://github.com/epam/edp-install/blob/master/deploy-templates/values.yaml#L289
• ensure documentation updated
• use https://icon-sets.iconify.design/material-symbols/monitoring/ icon

Steps:

• check kind: EDPComponent with type: grafana - if CR exist - show Link in Headlamp UI;
• get EDPComponent.grafana.spec.url (for example : https://grafana.example.com/)

Release 0.9.0

Acceptance Criteria:

• edp-headlamp 0.9.0 released;

Headlamp should contain configuration section for Jira integration

In headlamp we have section (like gitserver, remote cluster) which contains credentials for Jira integration

Acceptance Criteria:

• user can manage Jira secret with Configuration section;
• inherit behavior as for secrets for registry/sonar/nexus integration;
• in EDP env can be create only ONE secret;
• user cannot redefine name "jira-ciuser";

Secret Structure:

apiVersion: v1
kind: Secret
metadata:
  name: jira-user
  namespace: <edp-project>
  labels:
   app.edp.epam.com/secret-type: jira
data:
  username: <base64_token>
  password: <base64_url>
type: Opaque

Release 0.8.0

Acceptance Criteria:

• edp-headlamp 0.8.0 released;

[CD Pipeline Page] Optimize data flow

As EDP User, I want to have functional to autofill all services to latest tag by one click

Acceptance Criteria:

• User can use autofil logic for all application to avoid manual choose version for each of applications;
• Add 'Latest' button line on screen
• When user click 'Latest' button:
• **for each of component go to CodebaseImageStream.spec.tags and get array of tags;
• **pick the last tag from array;
• **put last tag it to dropdown;

Add DependencyTrack Configuration page

This task is the copy of Sonar approach, we need to add new section in configuration

Acceptance criteria:

A new tab in the configuration section with the name DependencyTrack
secret name: ci-dependency-track
secret data field: token
label: app.edp.epam.com/secret-type=dependency-track
Put dependencytrack tab near defectdojo

Use branding from: https://github.com/DependencyTrack/branding 

Refact configuration section

Clarify description for gitops repository

Acceptance Criteria:

is case if something wrong use can understand potential reason from error message on gitops section;
status of gitOps repository available like in gitservers
git server - mandatory field in marketplace section (please add autofill and enable required flag);

Hide the DEPLOY button in Headlamp UI for the Jenkins deploy scenario

In Jenkins deploy scenario, the DEPLOY button in the CD Pipeline details page doesn't work since the logic of the button implies triggering Argo CD, which impossible when we use Jenkins.

So it's desired to hide the button or make it gray (inactive) then.

Acceptance Criteria:

• In Jenkins deploy scenario Deploy button inactive OR return link to proper cd-pipeline in Jenkins;

Redesign CD pipeline page structure

Currently the page with CD pipelines is quite overloaded with information, let's make an approach so that there is one combined page to understand the nesting structure of CD pipelines and also a page for each CD stage with a detailed display of data.

Acceptance Criteria:

• EDP user can open a separate page to display detailed information for each CD stages;
• CD pipelines page contain information about main pipeline structure, without detailed information about CD stages;

Update GitServer provisioning

Acceptance Criteria:

• git server can be presented only one;
• git server status can be seen;
• secret name - hardcode - ci-gitlab/github;
• gerrit - without modification;

MarketPlace Page empty

STR:

Open EDP in incognito mode
Go to the Marketplace tab

AR: Marketplace tab is empty

ER: Should be displayed all available Marketplaces

Hide Create button for Component in case of git server do not exist

Merge codebase info and application info sections on project creating dialog

Take out separately the options for choosing the type of project and the available strategy.

Implement 'default branch' autofill (main by default). And move default branch options to 'Advanced Settings' sections in create project dialog.
Rename 'Application name' field to 'Component name'.
'Git Server' and 'Git repo relative path' must be in one line to improve user experience.
Implement button with icons for 'Build tool' field instead of drop-down.
Implement 'Project name' autofill and get value from 'Repository URL' parameter.

Add description for new 'Create project' dialog:

Application: Used to deploy services. It includes configuration files, deployment scripts, and other resources needed to create and manage the application's infrastructure.

Library: Provides reusable code that can be incorporated into services. It includes additional functions, modules that might be shared across services.

Autotests: Onboard and start defining Quality Gate for deployment pipelines here...

Create from template: Create a sample project from a template to start quickly.

Import project: Onboard your existing code to the EDP platform.

Clone project: Clone code from third-party VCS providers.

Fix latest, stable image stream buttons, fix adding registry secrets

STR:

Go to EDP configuration screen on Registry tab
Remove all external secrets
Add new one

ER: User should be able to add new secrets

AR: Create service account button is inactive

Detailed information isn't displayed for already created secrets

In other cases in no secret message tab name starts with a capital letter.

STR:

Create and build any application
Create CD pipeline
During deployment select application and press Latest button

ER: Latest image stream version should be selected

AR: Image stream version is empty, same problem for stable button

Redesign EDP Components page

This is new section in 'Configuration' section. Main goal - manage EDP component with EDP Portal.

Acceptance Criteria:

EDP user can manage (create/update/delete) EDP Components via Configuration section in EDP Portal;
align with gitserver approach;

Add logs viewer and terminal into stage applications deploy table columns

Add link to edp-gitops repo

Fix going back to previous page on resource deletion success

STR:

Create any application codebase
Create CD Pipeline using the created codebase
Open created pipeline
Delete pipeline from the inside

AR: User stayed on the same screen.

ER: User should be redirected to the previous screen

Add SSO secret management to EDP Portal Configuration section

Acceptance Criteria:

align with Nexus/Sonar/DD approach;
provider: Keycloak;

TBD: do we need to add URL field to secret?

Git Server. Git Ops issues

• fix git server creation(sshPort, httpsPort number type)
• fix crash on gitops page when there is no git server
• show no git servers message when there is no git server

Get environment namespace from cd stages

Update headlamp to 0.19.1

Recently, we've updated headlamp to 0.19.0 which has issues, so we've rollback'ed it to previous version with appropriate code changes. We have to update headlamp to 0.19.1 and bring back those code changes.

Reorganize documentation urls

[Configuration] Align pages to similar behaviour

We have configuration section with multiple similar looking pages. We should create a reusable component that is used for showing configuration pages with given parameters.

As a EDP User, I want to have capability to define values.yaml file per cd stage (environment)

Acceptance Criteria:

• EDP user can manage values as gitops approach;
• use EDP Portal to modify and save custom values.yaml;
• user can use approach to deploy envs with custom values;

language: helm
framework: gitops
build_tool: helm

Example Argo CD application with custom values:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: mm-fastapi-custom
spec:
  destination:
    name: in-cluster
    namespace: demo-deploy-dev
  project: demo
  sources:
  - repoURL: ssh://[email protected]:30222/edp-gitops
    targetRevision: main
    ref: values
  - helm:
      valueFiles:
        - $values/deploy/dev/mm-fastapi-values.yaml
      parameters:
        - name: image.tag
          value: 0.1.0-SNAPSHOT.2
        - name: image.repository
          value: registry.example.com/demo/mm-fastapi
      releaseName: mm-fastapi
    path: deploy-templates
    repoURL: ssh://[email protected]:30222/mm-fastapi
    targetRevision: build/0.1.0-SNAPSHOT.2
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

Template:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: mm-fastapi-custom
spec:
  destination:
    name: in-cluster
    namespace: demo-deploy-dev
  project: demo
  sources:
  - repoURL: ssh://[email protected]:30222/edp-gitops
    targetRevision: main
    ref: values
  - helm:
      valueFiles:
        - $values/<cdpipelinename>/<cdstagename>/<application-name>-values.yaml
      parameters:
        - name: image.tag
          value: 0.1.0-SNAPSHOT.2
        - name: image.repository
          value: registry.example.com/demo/mm-fastapi
      releaseName: mm-fastapi
    path: deploy-templates
    repoURL: ssh://[email protected]:30222/mm-fastapi
    targetRevision: build/0.1.0-SNAPSHOT.2
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

Pattern to provision edp-gitops repository:

apiVersion: v2.edp.epam.com/v1
kind: Codebase
metadata:
  labels:
    app.edp.epam.com/codebaseType: system
  name: edp-gitops
spec:
  buildTool: helm
  ciTool: tekton
  defaultBranch: main
  deploymentScript: helm-chart
  description: Custom values for deploy applications
  emptyProject: false
  framework: gitops
  gitServer: gerrit # can be depend on gitserver
  gitUrlPath: <path_to_repo>/edp-gitops
  jenkinsSlave: null
  jiraIssueMetadataPayload: null
  jobProvisioning: null
  lang: helm
  strategy: create
  ticketNamePattern: null
  type: system
  versioning:
    startFrom: 0.1.0-SNAPSHOT
    type: edp

EDP Portal steps:

• Add CD pipeline (GitOps) in configuration page;
• in CD pipeline (GitOps) configuration user can provide only path to repo;
• user cannot modify path after apply;
• after apply describe short information about codebase provisioning (repo path);
• Add custom values checkbox in cd stage;
• If user set checkbox with custom values to true - EDP portal create kind: Application with additional values path;
• when user provide path to codebase with custom values - user can create CD pipeline and new codebase must contain type app.edp.epam.com/codebaseType: system and name edp-gitops;
• hide deletion button if codebase contain app.edp.epam.com/codebaseType: system;
• if user do not configure CD pipeline (GitOps) in Configuration section - use cannot create CD pipeline in Environment page;
• add lock icon (System Repository) if have app.edp.epam.com/codebaseType: system label;
• Git Repo relative path available only for github, gitlab integration;
• add hardcoded field Repository Name;

Repository schema:
cdpipelinename/сdstagename/application-name-values.yaml

EDP-gitops repository must be unique per edp tenant

Release 0.7.0

Acceptance Criteria:

• edp-headlamp 0.7.0 released;

Add ECR Registry into registry configuration

apiVersion: v1
kind: ServiceAccount
metadata:
name: tekton
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::093899590031:role/AWSIRSACoreSandboxEdpDeliveryKaniko

apiVersion: v1
kind: ServiceAccount
metadata:
name: tekton
annotations:
eks.amazonaws.com/role-arn: <ARN_VALUE>

Add link to values.yaml file from CD Stage in edp-gitops repo

For gitlab/github example:
https://git.epam.com/epmd-edp/sk-dev/edp-gitops/blob/main/demo/qa/goapp-values.yaml

https://git.epam.com/epmd-edp/sk-dev/edp-gitops/blob/main/<pipeline_name>/<stage_name>/<codeabase_name>-values.yaml

Or open just link to repo in main branch.

[helm]Move keycloakUrl under oidc section

keycloakUrl is used only when oidc is enabled, so let's move it to oidc section