tryweirdier/xmake-cloud-helm

xmake Cloud Helm Charts

Nx Cloud Helm Chart

A lot of organizations deploy Nx Cloud to Kubernetes.

This repo contains:

• Nx Cloud Helm Chart
• Instructions on how to install Nx Cloud using Helm
• Instructions on how to install Nx Cloud using kubectl. See here.

Installing Using Helm

Steps:

1. Deploy MongoDB Kubernetes Operator
   • using helm: https://github.com/mongodb/helm-charts
   • using kubectl: https://github.com/mongodb/mongodb-kubernetes-operator
2. Create a mongodb replica set
3. Create a secret
4. Install Nx Cloud using helm

Step 1: Deploy MongoDB Kubernetes Operator

If you are using a hosted MongoDB installation (e.g., Mongo Atlas or CosmosSB, or you are running one yourself), you can skip steps 1 and 2.

> helm repo add mongodb https://mongodb.github.io/helm-charts
> helm install community-operator mongodb/community-operator

Step 2: Deploy a MongoDB replica set

> kubectl apply -f examples/mongodb.yml

This will create a secret. You can get the value of the secret as follows:

> kubectl get secret cloud-mongodb-nrwl-api-admin-user -o go-template='{{range $k,$v := .data}}{{"### "}}{{$k}}{{"n"}}{{$v|base64decode}}{{"nn"}}{{end}}'

You might need to wait a bit for the Pods to be created before this secret will be available.

The result should look like this: mongodb+srv://admin-user:[email protected]/nrwl-api?replicaSet=cloud-mongodb&ssl=false .

Extract the connection string and paste it into your secret.yml.

Step 3: Create a secret

Create a secret by running kubectl apply -f examples/secret.yml

Step 4: Install Nx Cloud using helm

> helm repo add nx-cloud https://nrwl.github.io/nx-cloud-helm
> helm install nx-cloud nx-cloud/nx-cloud --values=overrides.yml

examples/overrides contains the min overrides files. You need to provision:

1. The image tag you want to install
2. nxCloudAppURL which is the url used to access ingress from CI and dev machines ( e.g., https://nx-cloud.myorg.com).
3. secret/name the name of the secret you created in Step 3.
4. secret/nxCloudMongoServerEndpoint, the name of the key from the secret. 5secret/adminPassword, the name of the key from the secret.

If you only applied the secret from Step 3, the only thing you will need to change is nxCloudAppURL.

Cloud Containers

The installation will create the following:

1. nx-cloud-frontend (deployment)
2. nx-cloud-api (deployment)
3. nx-cloud-nx-api (deployment)
4. nx-cloud-file-server (deployment)
5. nx-cloud-aggregator (cron job)

Ingress, IP, Certificates

You can configure Ingress. For instance, the following will see the ingress class to 'gce', the global static ip name to 'nx-cloud-ip', and will set a global Google managed certificate.

image:
  tag: 'latest'

nxCloudAppURL: 'https://nx-cloud.myorg.com'

ingress:
  class: 'gce'
  globalStaticIpName: 'nx-cloud-ip'
  managedCertificates: 'cloud-cert'

secret:
  name: 'cloud'
  nxCloudMongoServerEndpoint: 'NX_CLOUD_MONGO_SERVER_ENDPOINT'
  adminPassword: 'ADMIN_PASSWORD'

This configuration will look different for you. You will have a different global static ip and your cert name will also be different. If you are interested in creating the two using GKE, check out the following links:

• Reserving a static external IP address
• Using Google-managed SSL certificates

If you aren't using GKE, ingress.class will also be different. For instance, this is a setup for EKS:

image:
   tag: 'latest'

nxCloudAppURL: 'https://nx-cloud.myorg.com'

ingress:
   class: 'alb'
   albScheme: 'internet-facing'
   albListenPorts: '[{"HTTPS":443}]'
   albCertificateArn: 'arn:aws:acm:us-east-1:411686525067:certificate/8adf7812-a1af-4eae-af1b-ea425a238a67'

secret:
   name: 'cloud'
   nxCloudMongoServerEndpoint: 'NX_CLOUD_MONGO_SERVER_ENDPOINT'
   adminPassword: 'ADMIN_PASSWORD'

If you need to have a detailed ignress configuration, you can tell the package to skip defining ingress:

image:
   tag: 'latest'

nxCloudAppURL: 'https://nx-cloud.myorg.com'

ingress:
    skip: true

and then define it yourself:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nx-cloud-ingress
  annotations:
     
  labels:
    app: nx-cloud
spec:
  rules:
    - http:
        paths:
          # define the next /file section only if you use the built-in file server
          - path: /file
            pathType: Prefix
            backend:
              service:
                name: nx-cloud-file-server-service
                port:
                  number: 5000
          - path: /nx-cloud
            pathType: Prefix
            backend:
              service:
                name: nx-cloud-nx-api-service
                port:
                  number: 4203
          - path: /api
            pathType: Prefix
            backend:
              service:
                name: nx-cloud-nrwl-api-service
                port:
                  number: 4000
          - path: /graphql
            pathType: Prefix
            backend:
              service:
                name: nx-cloud-nrwl-api-service
                port:
                  number: 4000
          - path: /auth
            pathType: Prefix
            backend:
              service:
                name: nx-cloud-nrwl-api-service
                port:
                  number: 4000
          - path: /download
            pathType: Prefix
            backend:
              service:
                name: nx-cloud-nrwl-api-service
                port:
                  number: 4000
  defaultBackend:
    service:
      name: nx-cloud-frontend-service
      port:
        number: 8080

External Secrets

If you want to store your secrets in an external tool (e.g., AWS Secret Manager), the easiest way to set it up is via External Secrets.

External Secrets Operator is a Kubernetes operator that integrates external secret management systems like AWS Secrets Manager, HashiCorp Vault, Google Secrets Manager, Azure Key Vault and many more. The operator reads information from external APIs and automatically injects the values into a Kubernetes Secret.

You can provision one of many secrets in the external tool, but all of them should be merged into a single K8s secret. And the target k8s secret name should match the one in your helm's values file.

Example:

---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: external
spec:
  refreshInterval: 10m
  secretStoreRef:
    kind: ClusterSecretStore
    name: external-secrets
  target:
    name: cloud
    creationPolicy: Owner
  dataFrom:
    - extract:
          key: GoogleCloudNxCloudSecret

Follow the instructions for your provider on how to set it up.

Variations

External File Storage

If you use AWS or Azure, you can configure Nx Cloud to store cached artifacts on S3 or Azure Blob. In this case, you won't need the PVC or the file-server container. S3 and Azure Blob also tend to be faster.

image:
  tag: 'latest'

nxCloudAppURL: 'https://nx-cloud.myorg.com'

awsS3:
  enabled: true
  bucket: 'nx-cloud'
  # accelerated: true  uncomment when using accelerated bucket
  # endpoint: ''  uncomment when using a custom endpoint

secret:
  name: 'cloudsecret'
  nxCloudMongoServerEndpoint: 'NX_CLOUD_MONGO_SERVER_ENDPOINT'
  adminPassword: 'ADMIN_PASSWORD'
  awsS3AccessKeyId: 'AWS_S3_ACCESS_KEY_ID'
  awsS3SecretAccessKey: 'AWS_S3_SECRET_ACCESS_KEY'

image:
  tag: 'latest'

nxCloudAppURL: 'https://nx-cloud.myorg.com'

azure:
  enabled: true
  container: 'nx-cloud'

secret:
  name: 'cloudsecret'
  nxCloudMongoServerEndpoint: 'NX_CLOUD_MONGO_SERVER_ENDPOINT'
  adminPassword: 'ADMIN_PASSWORD'
  azureConnectionString: 'AZURE_CONNECTION_STRING'

Note that the secret must contain AWS_S3_ACCESS_KEY_ID, AWS_S3_SECRET_ACCESS_KEY or AZURE_CONNECTION_STRING.

GitHub Auth

To use GitHub for user authentication, you can use the following configuration:

image:
  tag: 'latest'

nxCloudAppURL: 'https://nx-cloud.myorg.com'

github:
  auth:
    enabled: true

secret:
  name: 'cloudsecret'
  nxCloudMongoServerEndpoint: 'NX_CLOUD_MONGO_SERVER_ENDPOINT'
  githubAuthClientId: 'GITHUB_AUTH_CLIENT_ID'
  githubAuthClientSecret: 'GITHUB_AUTH_CLIENT_SECRET'

Note that the secret must contain GITHUB_AUTH_CLIENT_ID and GITHUB_AUTH_CLIENT_SECRET. Read here on how to get those values.

GitHub Integration

To enable the GitHub PR integration, you can use the following configuration:

image:
  tag: 'latest'

nxCloudAppURL: 'https://nx-cloud.myorg.com'

github:
  pr:
    enabled: true
    # apiUrl: '' uncomment when using github enterprise 

secret:
  name: 'cloudsecret'
  nxCloudMongoServerEndpoint: 'NX_CLOUD_MONGO_SERVER_ENDPOINT'
  githubWebhookSecret: 'GITHUB_WEBHOOK_SECRET'
  githubAuthToken: 'GITHUB_AUTH_TOKEN'

Note that the secret must contain GITHUB_WEBHOOK_SECRET and GITHUB_AUTH_TOKEN. Read here on how to get those values.

More Information

You can find more information about Nx Cloud and running it on prem here.