cloudflare/helm-charts

Cloudflare Helm Charts

About

A convenient location to publish Cloudflare helm charts

Setup

helm repo add cloudflare https://cloudflare.github.io/helm-charts
helm repo update

Discovery

helm search repo cloudflare

Contents

• charts/cloudflare-tunnel: Helm 3 chart using cloudflared best practices

helm-charts's Issues

support securityContext

without it the pod will fail in a restricted environment:

  Warning  Failed          0s (x3 over 11s)  kubelet, 10.246.3.13  Error: container has runAsNonRoot and image will run as root

Validation failed while installing

Hello

I am getting the following error when installing:

helm install --name test -n testns --set rbac.create=true --set controller.ingressClass=argo-tunnel --set controller.logLevel=6 cloudflare/argo-tunnel --debug

[debug] Created tunnel using local port: '41079'
[debug] SERVER: "127.0.0.1:41079"
[debug] Original chart version: ""
[debug] Fetched cloudflare/argo-tunnel to /home/smx/.helm/cache/archive/argo-tunnel-0.6.5.tgz
[debug] CHART PATH: /home/smx/.helm/cache/archive/argo-tunnel-0.6.5.tgz

Error: validation failed: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1"

feat: Allow users to inject credentials file separately

Helm charts certainly make things easier and more standardised, so I am glad this exists. My suggestion/feature request would be to allow users to inject the credentials file from a secret value, as at the moment the only way to pass the tunnel secret is through setting .Values.cloudflare.secret.

This is not ideal because:

a. It requires the whole chart to be a secret, which in many cases it might not be (example: if it just lives in Github)
b. Doesn't give flexibility for users to use external secrets managers

A couple of solutions are possible:

1. Encoding the whole local values.yaml as a secret

Correct if I am wrong, but this seems to be the only solution with the current approach (raw secret in the values.yaml). ExternalSecrets users can leverage the templating engine to create a values.yaml that then is used by helm (well, at least with Flux it is possible to reference secrets as values).

If this is the way to go, perhaps might be good to leave some notes in the README on the safest way of doing this.

2. Change cloudflared to accept all credentials values via env variables and allow env injection in the chart

This is quite a common approach in other projects but probably would require refactoring not only this chart, but as well cloudflared. Possibly unlikely?

3. Chart accepts a credentialsSecretRef

In this case, users can map the credential file secret somewhere else and, the secrets.yaml shouldn't be used if this is set. Instead, the deployment.yaml would load the secret defined by the user.

This would be very handy, not attaching this project to any specific secret management solution, while still providing users standardisation through Helm.

How to properly authenticate to Cloudflare?

I'm trying to use the cloudflare-tunnel helm chart, but I'm not sure how to configure the below options properly:

cloudflare:
  # Your Cloudflare account number.
  account: ""
  # The secret for the tunnel.
  secret: ""

Could you provide instructions on how to get Cloudflare account number and secret for the tunnel ?
I can't find any documentation or example on how to use it properly.

Thanks in advance for any help :)

Willing to maintain

@obezuk I am willing to maintain this project in order to pull some of the responsibility off your plate. I maintained a private cloudflare tunnel helm chart for a previous employer (not a huge accolade, I know) but would be willing to parse through these PRs, keep the helm charts updated and stable, and the repository clean and moderated.

• Can monitor and parse issues
• Can aid in PR approval and merging
• Can aid in moderating forward progress for the repository
• Can use actions to check new versions of the tunnel and update helm chart when cloudflared has a new version. This would allow us to keep the helm chart in line with the application and would keep helm packages current
• Willing to spend time monthly checking on the repository
• Would fix #66

cloudflare-tunnel-remote chart secret

This chart forces me to put a hard-coded and sensitive value to my values.yaml file. I can't externalize this secret as an externalSecret object.
Proper way of creating this secret should be optional, if it exists don't create, if it doesn't exist then create it. Also parameterize the secret name.
Same problem applies to cloudflare-tunnel chart ingress.[].originrequest.access.audTag.

Neither of these charts look professionally written.

Difference between cloudflare-tunnel and cloudflare-tunnel-remote?

I see there are now two helm-charts in this repo.

cloudflare-tunnel doesn't seem to manage the public hostnames in cloudflare automatically, which means they have to be managed remotely anyway.

cloudflare-tunnel-remote simply errors with:

rror: unable to build kubernetes objects from release manifest: error validating "": error validating data: ValidationError(Deployment.spec.template.spec.containers[0].envFrom[0].secretRef): unknown field "key" in io.k8s.api.core.v1.SecretEnvSource

Some aadditional documentation on how these are used and if these charts are ready for production use would be great.

Missing pod labels/annotations

Thing that this chart is missing and could be valuable:

• Adding custom tags/labels/annotations to the deployment

Any plans on making this the official helm charts repo?

Hi,

When looking for a cloudflared helm chart on Artifact Hub, I could not find the ones you have here: https://artifacthub.io/packages/search?ts_query_web=cloudflared&sort=relevance&page=1

It feels like there is currently no official helm chart for cloudflared.

Are there any plans to make the charts you have here the official ones and maybe adding some documentation?

Thanks!

External secrets for cloudflare-tunnel-remote

It does not seem to possible to create the tunnel_token as a secret separate from the values.yaml file as #38 has implemented for the Cloudflare-tunnel chart so that it can be stored securely (eg in GitHub)

Protocol config option

Evidently a lot of people are still have quic connectivity problems after 2022.3.4 (myself included) that they cannot do much about themselves as it's ISP related most of the time and in k8s case pods never reach http2 when set to auto if quic does not connect, instead they enter a crash loop, so it would make sense to include an optional protocol config option as a remedy.

Missing TopologySpreadConstraints

This chart does not include a TopologySpreadConstraints which should be used for deployments in production to spread out pods across nodes and zones. It should be aware of different revisions by using matchLabelKeys

cloudflare-tunnel Helm chart doesn't work!

cloud-flare Helm chart - version: 0.3.0 - when deployed, fails!

Here's the error message;

    2023-10-17T02:06:55Z ERR Failed to serve quic connection error="Unauthorized: Invalid tunnel secret" connIndex=0 event=0 ip=198.41.192.272
    023-10-17T02:06:55Z ERR Register tunnel error from server side error="Unauthorized: Invalid tunnel secret" connIndex=0 event=0 ip=198.41.192.272

This message prevents the pod to deploy.

Pod	Ready	Status	Restarts
cloudflare-tunnel-8594d78b6f-d82bd	0/1	CrashLoopBackOff	7

Is this supported?

Reading the Cloudflare docs, it seems like this is no longer supported. If so, you should put something in the README to state this clearly. It appears like Cloudflare offers no support for connecting Kubernetes clusters (apart from docs). Is this correct? If so, why? Seems like a major use case?

Upgrade cloudflared version

The current cloudflared version is over a year out of date and needs to be updated to a modern version.

Related to https://github.com/cloudflare/helm-charts/pull/37/files

Add network polices to helm chart

Consider adding network policies as an opt-in feature as part of this helm chart.

With some templating - this could be handled gracefully and configurable.

considerations:

default deny (baseline deny all):

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress
  ingress: []
  egress: []

dns-egress (allowing ingress options to target k8s dns IE nginx.nginx.svc.cluster.local:443)

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns-egress
spec:
  podSelector: {}
  policyTypes:
  - Egress
  # Allow access to DNS
  egress:
  - to:
    - namespaceSelector: {}
    ports:
    - port: 53
      protocol: UDP

egress to target pods:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-tunnel-egress
spec:
  podSelector: {}
  policyTypes:
  - Egress
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: nginx
      podSelector:
        matchLabels:
          app: nginx
    ports:
    - port: 8443

public egress (to cloudflare - still needs some more definitive targeting)

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-public-egress
spec:
# https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/install-and-setup/ports-and-ips/
  podSelector: {}
  policyTypes:
  - Egress
  - Ingress
  egress:
  - ports:
    - protocol: TCP
      port: 443
    - protocol: UDP
      port: 7844
    - protocol: TCP
      port: 7844
    to:
    - ipBlock:
        cidr: 0.0.0.0/0

This is currently working for my test - hopefully I haven't overlooked anything critical.

Corporate Proxy

If the zero tunnel is possible behind a Corporate Proxy it would be awsome to configure that inside the values.yaml

I tryed to modify the depmoyment yaml with http_proxy and https_proxy env but this doesnt work.

2023-07-04T14:28:39Z INF Starting tunnel tunnelID=ID
2023-07-04T14:28:39Z INF Version 2023.5.1
2023-07-04T14:28:39Z INF GOOS: linux, GOVersion: go1.19.9, GoArch: amd64
2023-07-04T14:28:39Z INF Settings: map[config:/etc/cloudflared/config/config.yaml cred-file:/etc/cloudflared/creds/credentials.json credentials-file:/etc/cloudflared/creds/credentials.json metrics:0.0.0.0:2000 no-autoupdate:true]
2023-07-04T14:28:39Z INF Generated Connector ID: ID
2023-07-04T14:28:39Z INF Initial protocol quic
2023-07-04T14:28:39Z INF ICMP proxy will use 10.244.1.103 as source for IPv4
2023-07-04T14:28:39Z INF ICMP proxy will use fe80::dc0f:10ff:fe56:27e2 in zone eth0 as source for IPv6
2023-07-04T14:28:39Z WRN The user running cloudflared process has a GID (group ID) that is not within ping_group_range. You might need to add that user to a group within that range, or instead update the range to encompass a group the user is already in by modifying /proc/sys/net/ipv4/ping_group_range. Otherwise cloudflared will not be able to ping this network error="Group ID 65532 is not between ping group 1 to 0"
2023-07-04T14:28:39Z WRN ICMP proxy feature is disabled error="cannot create ICMPv4 proxy: Group ID 65532 is not between ping group 1 to 0 nor ICMPv6 proxy: socket: permission denied"
2023-07-04T14:28:39Z INF Starting metrics server on [::]:2000/metrics
2023-07-04T14:28:39Z WRN Your version 2023.5.1 is outdated. We recommend upgrading it to 2023.6.1
2023-07-04T14:28:44Z ERR Failed to create new quic connection error="failed to dial to edge with quic: timeout: no recent network activity" connIndex=0 event=0 ip=198.41.192.7
2023-07-04T14:28:44Z INF Retrying connection in up to 2s connIndex=0 event=0 ip=198.41.192.7
2023-07-04T14:28:49Z INF Initiating graceful shutdown due to signal terminated ...
2023-07-04T14:28:49Z INF Tunnel server stopped
2023-07-04T14:28:49Z INF Metrics server stopped

Changed Deployment.yaml

.....
      containers:
        - name: cloudflare-tunnel
          image: cloudflare/cloudflared:2023.5.1
          args:
            - tunnel
            - '--config'
            - /etc/cloudflared/config/config.yaml
            - run
          env:
            - name: http_proxy
              value: http://myproxy.local:3128
            - name: https_proxy
              value: http://myproxy.local:3128
            - name: HTTP_PROXY
              value: http://myproxy.local:3128
            - name: HTTPS_PROXY
              value: http://myproxy.local:3128
            - name: NO_PROXY
              value: >-
                10.0.0.0/8,.svc,192.168.0.0/16,127.0.0.1,172.16.0.0/12
....

It looks fine because the proxy port is used but also 7844 which are dropped because its not the proxy...

Replace argo-tunnel chart

This repository is an official helm chart repo and current argo-tunnel chart in here is a legacy which is not supported anymore by Cloudflare. Mean while in this repo which is also belongs to cloudflare there's a helm-chart which is fully functional, and we "SIA Setupad" as your enterprise members are using it.

The only problem there's no way to install that chart through helm because it has to be stored in cloudflare/helm-charts repo, naming format is crucial for git repo in order for it to work as helm repo. So I offer to move that helm-chart to here, and replace the current one. Would you accept some PR? I've also asked tons of people through our account managers and they advice to make a PR in here first.

@mattalberts as a main contributor I address this to you. We are willing to contribute we just need your blessing in here.

Also I've been able to deploy cloudflare tunnels with that other helm-chat and I can add examples for it.

Public hostnames aren't propagated automatically

Public hostnames aren't propagated automatically. In the example below you can see our configuration:

cloudflare:
  ingress:
    - hostname: prom-server.domain.com
      service: https://prometheus-server.monitoring:80

But these settings aren't applying automatically. It means that I still have to add public hostnames in Tunnel configuration manually. Are there any additional settings required? Or this functionality isn't working yet?

The 0.6.1 release doesn't show up in helm

I am still seeing 0.6.0 showing up after a helm repo update and helm search

using old apiVersion: extensions/v1beta1 Deployment resource definition

which has been deprecated since Kubernetes 1.16
https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/

Will chart be updated to current versions?

Chart is 4-5 years old. If this is a official helm repo it is way behind on everything.

It is under the cloudflare github account and the README.md stats "A convenient location to publish Cloudflare helm charts".

Is this project being actively maintained?

There's a number of issues and pull requests that are multiple months old. Not sure who can approve PRs?

Can a CODEOWNERS file be setup to automatically tag the right people for review?

Missing PodDisruptionBudget (PDB)

This chart does not include a PodDisruptionBudget (PDB) which should be used for deployments in production

please add (nodeSelector or affinities) and tolerations to the deployment template

in order to have better control over where the tunnel deployment's pod end up, please allow:

• nodeSelector or affinities
• tolerations

to be specified
in
https://github.com/cloudflare/helm-charts/blob/main/charts/cloudflare-tunnel-remote/templates/deployment.yaml

Support extraVolumes and extraVolumeMounts

I have a use case where I inject the K8s secret with Kubernetes Secrets Store CSI Driver.

To be able to do this, I need to have extra Volumes or extra Volume Mounts (optional) for the deployment.

I'll be happy to open the PR if people think it is useful.

Configurable default ingress service http_status:400

I was trying to use my own ingress service instead of http_status:400. There is no way to do that with current implementation. Either add default service wtih ingress map or separate variable to change or modify.

helm-charts/charts/cloudflare-tunnel/templates/configmap.yaml

Line 29 in b85225e

- service: http_status:404

Below is what I do now to update value on the fly and after remove pods.

kubectl get cm/cloudflare-tunnel -n cloudflare-tunnel -o yaml |
        sed 's/http_status:404/http:\/\/nginx-ingress-controller.nginx-ingress.svc.cluster.local.:80/g' |
        kubectl apply -f -

kubectl -n cloudflare-tunnel delete pods --all

Would have been better if there was way to modify default ingress service or include that as default for ingress service map.