f5networks/charts

Helm charts for using F5 products and services in Kubernetes and OpenShift environments.

F5 Networks Helm Charts

This repository contains helm charts for use with some F5 Networks products and services within a Kubernetes or OpenShift environment.

Note: Charts may require access to kube-system namespace and/or cluster wide permissions for full functionality. Be sure to install/configure helm/tiller appropriately.

---

Deprecation Notice:

This repository is no longer active, Please refer k8s-bigip-ctlr for latest updates.

---

Stable Charts

The stable directory contains charts that are created/curated and tested by F5 Networks. These charts are supported by F5 Networks (see SUPPORT for details).

To add the stable repo to helm:

helm repo add f5-stable https://f5networks.github.io/charts/stable

Stable Charts:

• f5-bigip-ctlr - Use this chart to deploy the k8s-bigip-ctlr in Kubernetes or OpenShift.
• f5-bigip-ingress - Use this chart to deploy Ingress resources in Kubernetes or OpenShift.

Documentation

Each chart has a README describing its basic functionality. The values.yaml file for each chart shows the default values and links to documentation for the resources the chart deploys.

Incubation Charts

The incubation charts may have been created by F5 Networks or by external contributors. These charts have not undergone full testing and are subject to change. F5 Networks does not provide technical support for templates in the incubation directory.

To access additional charts in a development or testing mode that may not be documented:

helm repo add f5-incubator https://f5networks.github.io/charts/incubator

charts's Issues

Cannot list resource for ExternalDNS and Policy: Forbidden user

Description

For bug (See diagnostic section for error details):
A few issues observed in using the latest image(2.7.0) with this latest helm chart(0.0.15):

1. The ExternalDNS CRD deployed from the helm chart (0.0.15) appears to have a different plural which the 2.7.0 image is referring to
2. This plural does not match the ExternalDNS CRD deployed from the helm chart
3. The clusterrole created for the K8s service account does not have the necessary privileges for that CRD:

- apiGroups:
  - cis.f5.com
  resources:
  - virtualservers
  - tlsprofiles
  - transportservers
  - externaldnss
  - ingresslinks
  - virtualservers/status
  - ingresslinks/status
  verbs:
  - get
  - list
  - watch
  - update
  - patch

4. The clusterrole is also missing "policies".
5. It appears the latest CRDs are not in this helm chart that are required for 2.7.0 image

Kubernetes Version

Version:

Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.10", GitCommit:"8152330a2b6ca3621196e62966ef761b8f5a61bb", GitTreeState:"clean", BuildDate:"2021-08-11T18:06:15Z", GoVersion:"go1.15.15", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.8", GitCommit:"5575935422cc1cf5169dfc8847cb587aa47bac5a", GitTreeState:"clean", BuildDate:"2021-06-16T12:53:07Z", GoVersion:"go1.15.13", Compiler:"gc", Platform:"linux/amd64"}

Controller Version

Version: 2.7.0 (image version)
Version. 0.0.15 (helm chart version)

BIG-IP Version

Version: BIG-IP 15.1.2.1 Build 0.0.10 Point Release 1

Helm / Tiller Version

helm: v3.7.2

Diagnostic Information

E0113 15:39:34.174391       1 reflector.go:138] github.com/F5Networks/k8s-bigip-ctlr/pkg/crmanager/informers.go:82: Failed to watch *v1.ExternalDNS: failed to list *v1.ExternalDNS: externaldnses.cis.f5.com is forbidden: User "system:serviceaccount:f5-bigip:f5-bigip-ctlr" cannot list resource "externaldnses" in API group "cis.f5.com" at the cluster scope
E0113 15:39:34.238987       1 reflector.go:138] github.com/F5Networks/k8s-bigip-ctlr/pkg/crmanager/informers.go:94: Failed to watch *v1.Policy: failed to list *v1.Policy: policies.cis.f5.com is forbidden: User "system:serviceaccount:f5-bigip:f5-bigip-ctlr" cannot list resource "policies" in API group "cis.f5.com" at the cluster scope
E0113 15:39:36.690098       1 reflector.go:138] github.com/F5Networks/k8s-bigip-ctlr/pkg/crmanager/informers.go:82: Failed to watch *v1.ExternalDNS: failed to list *v1.ExternalDNS: externaldnses.cis.f5.com is forbidden: User "system:serviceaccount:f5-bigip:f5-bigip-ctlr" cannot list resource "externaldnses" in API group "cis.f5.com" at the cluster scope
E0113 15:39:37.222251       1 reflector.go:138] github.com/F5Networks/k8s-bigip-ctlr/pkg/crmanager/informers.go:94: Failed to watch *v1.Policy: failed to list *v1.Policy: policies.cis.f5.com is forbidden: User "system:serviceaccount:f5-bigip:f5-bigip-ctlr" cannot list resource "policies" in API group "cis.f5.com" at the cluster scope

[feature request] Please move Charts to helm/charts

Description

The Helm Charts repo is a good place to put these charts.

Please see the contributing guidelines for details on getting the process started.
https://github.com/helm/charts/blob/master/CONTRIBUTING.md
I am also happy to help with the process.

Kubernetes Version

<Version of Kubernetes / OpenShift being used>

Controller Version

BIG-IP Version

Helm / Tiller Version

<Version of Helm / Tiller being used>

Diagnostic Information

<Configuration files, error messages, logs>
Note: Sanitize the data. For example, be mindful of IPs, ports, application names and URLs
Note: The following F5 article outlines the information required when opening an issue.
https://support.f5.com/csp/article/K60974137

The f5-bigip-ctlr helm chart lists version as optional but there's no default value (thus its not optional)

The big-ip-ctlr helm chart readme lists the version as optional:

version	Optional	CIS Controller image tag	latest

However if you omit setting it the chart won't install with an error indicating invalid image name:

  Warning  InspectFailed  9m38s (x13 over 11m)  kubelet, d-cont-wkr1  Failed to apply default image tag "f5networks/k8s-bigip-ctlr:": couldn't parse image reference "f5networks/k8s-bigip-ctlr:": invalid reference format
  Warning  Failed         6m50s (x26 over 11m)  kubelet, d-cont-wkr1  Error: InvalidImageName

Setting the version in the helm chart does address the problem but the readme shouldn't say its optional and/or the default values.yml should be setting it to latest

[bug] k8s-bigip-ctlr boolean flags not set correctly

Description

Using the following args in the values.yaml file:

args:
  bigip_url: https://<URL>/
  bigip_partition: <PARTITION>
  log_level: DEBUG
  pool_member_type: cluster
  insecure: true                 
  manage-ingress: false           
  custom-resource-mode: true      
  log-as3-response: true          
  ipam: false

Results in the following rendered deployment.yaml file:

args:
- --credentials-directory
- /tmp/creds
- --bigip-partition
- "<PARTITON>"
- --bigip-url
- "<URL>"
- --custom-resource-mode
- "true"
- --insecure
- "true"
- --ipam
- "false"
- --log-as3-response
- "true"
- --log-level
- "DEBUG"
- --manage-ingress
- "false"
- --pool-member-type
- "cluster"

This looks OK but the problem is that the boolean flags are now added as args to the k8s-bigip-ctlr command in the following way (end result):

/app/bin/k8s-bigip-ctlr --ipam false

I've omitted the rest of the flags to highlight the problem, --ipam is a boolean flag which means that it needs to be set as follows to be set to false:

--ipam=false

The way it works right now is that false is just another argument to k8s-bigip-ctlr and not setting the --ipam flag to false. When adding ipam: false to the args list makes it evaulate to true instead, the reversed behavior. This flag is set to false by default. They way boolean flags are handled is the expected behavior of the pflag package used in k8s-bigip-ctlr, see the resources/links below.

To fix this we could change:

charts/src/stable/f5-bigip-ctlr/templates/f5-bigip-ctlr-deploy.yaml

Lines 49 to 52 in 26590a4

	{{- range $key, $value := .Values.args }}
	- --{{ $key | replace "_" "-"}}
	- {{ $value | quote }}
	{{- end }}

to this instead:

{{- range $key, $value := .Values.args }}
- --{{ $key | replace "_" "-"}}={{ $value }}
{{- end }}

I've removed the quote function to not pass the options as strings where boolean is expected. Using the change above makes the rendered args look like this:

args:
- --credentials-directory
- /tmp/creds
- --bigip-partition=<PARTITION>
- --bigip-url=https://<URL>
- --custom-resource-mode=true
- --insecure=true
- --ipam=false
- --log-as3-response=true
- --log-level=DEBUG
- --manage-ingress=false
- --pool-member-type=cluster

This does work as expected and looking at the test suites in the k8s-bigip-ctlr the tests runs with flags set as --<FLAG>=<OPTION>, although not all command line flags or combinations of flag + option(s) are tested.

Resources that explains how the pflag behaves in regards to boolean flags:

• https://github.com/spf13/pflag#command-line-flag-syntax
• spf13/cobra#613 (comment)

Kubernetes Version

1.18.8

Controller Version

2.2.3

BIG-IP Version

14

Helm / Tiller Version

3.3.1

Diagnostic Information

N/A

add f5-bigip-ctlr chart to stable

Add stable chart for deploying the k8s-bigip-ctlr once developed / alpha testing is completed.

Deployment Template missing Resources

The values file has a section on setting resources for the controller but the deployment template does not seem to have it. This means we cannot set resources on the controller container.

Verify if lint errors for helm 2.8.1 are valid for use of helm generated helper functions

Using most recent helm (2.8.1) helm lint fails when rendering _helper.tpl as generated by helm create foo.

==> Linting src/incubator/f5-bigip-ctlr
[INFO] Chart.yaml: icon is recommended
[ERROR] templates/: render error in "f5-bigip-ctlr/templates/f5-bigip-ctlr-serviceaccount.yaml": template: f5-bigip-ctlr/templates/_helpers.tpl:6:31: executing "f5-bigip-ctlr.name" at <.Values.nameOverride>: map has no entry for key "nameOverride"

Error: 1 chart(s) linted, 1 chart(s) failed
1
src/incubator/f5-bigip-ingress
==> Linting src/incubator/f5-bigip-ingress
[INFO] Chart.yaml: icon is recommended
[ERROR] templates/: render error in "f5-bigip-ingress/templates/f5-bigip-ingress.yaml": template: f5-bigip-ingress/templates/_helpers.tpl:15:14: executing "f5-bigip-ingress.fullname" at <.Values.fullnameOver...>: map has no entry for key "fullnameOverride"```

Move `f5-bigip-ingress` Helm chart to networking.k8s.io/v1 Ingress API

Description

The extensions/v1beta1 and networking.k8s.io/v1beta1 API versions of Ingress is no longer served as of v1.22.

Migrate manifests and API clients to use the networking.k8s.io/v1 API version, available since v1.19.

reference

[Feature Request] Remove need for Admin, Add Token Authentication for K8S-Bigip API REST calls

What chart am I using:
kubernetes bigip management (v2)
What version of Bigip am I using:
12.1.1
What version of the chart/images am I using
1.5.x
What is the feature you want
When using the kubernetes integration for bigip we need to:

• Restrict user rights to a specific partition
• Use a partition level role (NOT ADMIN!!!)
• Enable token-auth support in the k8s solution (admin is the only basic-auth accessor on the REST API)

Why is this important?

• A kubernetes cluster should not have free access to the bigip outside of it's own partition.

Update the included CRDs

I think the included CRDs version should be upgraded to reflect the latest CRDs in version 2.16.0

[bug] not all args set in values.yaml forwarded correctly

Description

The helm chart for the f5-bigip-ctrl does not pass all additional args to the controller as desired, since it wraps them in additional quotes:
https://github.com/F5Networks/charts/blob/master/src/stable/f5-bigip-ctlr/templates/f5-bigip-ctlr-deploy.yaml#L63
This results in using the default values for example for use-node-internal argument, making the controller always export internal IPs to the f5 device, even though configured otherwise in the values.yaml.
Changing the args forwarding in f5-bigip-ctrl-deploy.yaml as follows solves the problem:

.
.
.
61    {{- range $key, $value := .Values.args }}
62            - --{{ $key | replace "_" "-"}}={{ $value }}
63            {{- end }}
.
.
.

Kubernetes Version

1.14.1

Controller Version

1.9.1

BIG-IP Version

v13

Helm / Tiller Version

v2.14.2

[RFE] Add possibility to schedule the BIG-IP controller to k8s controller/master nodes

Description

Today the Deployment manifest template of the F5 BIGIP controller helm chart doesn't have a way of forcing the controller to be scheduled on controller/master nodes. There's two opinonated reasons for this:

• Make sure controller workloads doesnt compete with the cluster resources used by "regular" workloads
• Although the BIGIP controller isnt vital for the k8s control plane it's vital for other reasons and should not be mixed with other workloads

One way of doing this is (what we've tested) is to add a combination of tolerations and node-selector parameters to the Deployment manifest template.

What we've added to test this so far:

nodeSelector:
  node-role.kubernetes.io/master: ""
tolerations:
- effect: NoSchedule
  key: node-role.kubernetes.io/master

Any input on this?

I could start off with a PR to add the possibility to configure this via values and toggle this within the Deployment manifest template.

Kubernetes Version

v1.18+

Controller Version

v2.2.3

BIG-IP Version

N/A

Helm / Tiller Version

v3.3.1

Diagnostic Information

N/A

use `--set` or `-f` for required values

For users it is better to have a chart fail to render (before attempting to deploy) if a required value -- e.g. the url of the BigIP device -- is not provided. This adds complexity when using helm lint.

[bug] RBAC resources are not unique and not optional

Description

With the current chart the RBAC resources are hard coded and not optional so it's impossible to deploy two instances of the Chart in the same namespace.

<For bug: Describe the bug in detail, observed versus expected behavior, steps to reproduce the issue, and a brief description of your deployment scenario>

Kubernetes Version

<Version of Kubernetes / OpenShift being used>

Controller Version

BIG-IP Version

Helm / Tiller Version

<Version of Helm / Tiller being used>

Diagnostic Information

<Configuration files, error messages, logs>
Note: Sanitize the data. For example, be mindful of IPs, ports, application names and URLs
Note: The following F5 article outlines the information required when opening an issue.
https://support.f5.com/csp/article/K60974137

[bug] serviceaccount in the helm chart is ignored for bigip f5 ctlr

namespace: kube-system
bigip_login_secret: f5-bigip-ctlr-login
serviceaccount: f5-bigip-ctlr01-serviceaccount
args:
...

When I try to install a second controller per the HA docs:

$ helm install -f f5-bigip-ctlr-01.values.yaml --name f5-controller-01 f5-incubator/f5-bigip-ctlr
Error: release f5-controller-01 failed: serviceaccounts "f5-bigip-ctlr-serviceaccount" already exists

And actually it looks like it's set in the helm chart but isn't setup correctly if I remove the whole thing:

status:
  replicas: 0
  observedGeneration: 1
  conditions:
    - type: ReplicaFailure
      status: 'True'
      lastTransitionTime: '2019-03-12T21:45:04Z'
      reason: FailedCreate
      message: >-
        pods "f5-controller-01-f5-bigip-ctlr-696cb64b55-" is forbidden: error
        looking up service account kube-system/f5-bigip-ctlr01-serviceaccount:
        serviceaccount "f5-bigip-ctlr01-serviceaccount" not found

EDIT:*
bigip_login_secret: f5-bigip-ctlr-login also seems broken as the secret must be named f5-bigip-ctlr-login.

[f5-bigip-ctlr] health check / probe missing

Readiness and Liveness Probe missing
https://clouddocs.f5.com/containers/v2/kubernetes/kctlr-app-install.html#run-health-checks

If not by default, should be enableable via the values file.

Add contribution guidelines

Repo lacks contribution guidelines and guidance on CLA requirements.

Migrate extensions/v1beta1 deploy to apps/v1

Description

Kubernetes 1.16 removes the extensions/v1beta1 api https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/. All templates using the deprecated APIs should be migrated to the stable API.

Kubernetes Version

N/A

Controller Version

N/A

BIG-IP Version

N/A

Helm / Tiller Version

N/A

Diagnostic Information

N/A

[bug] using args.partition causes unknown flag

Description

When running below command to launch chart to deploy the controller, k8s controller pod goes from "Error" to "CrashLoopBackOff " and never gets created.

helm install --set args.bigip_url=1.2.3.4 --set args.partition=kubernetes --set bigip_login_secret=bigip-login --set serviceaccount=k8s-bigip-ctlr f5-stable/f5-bigip-ctlr

Repro:

1. Install k8s
2. setup flannel and setup BIG-IP dummy node
3. setup bigip serviceaccount as necessary
4. setup secret for bigip login
5. install helm client and tiller
6. run helm install command as shown above using parameter "args.partition"
7. workaround is to change the parameter name to "bigip_partition"

Expected Behavior:
Pod gets created

Actual Behavior:
error

Request: please update the documentation if bigip_partition should be used as the parameter name when using the --set option.

Kubernetes Version

f5user@k8s-master:~$ kubectl version
Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState:"clean", BuildDate:"2018-10-05T16:46:06Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState:"clean", BuildDate:"2018-10-05T16:36:14Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}

Controller Version

f5user@k8s-master:~$ kubectl logs auxiliary-meerkat-f5-bigip-ctlr-5877c74b9b-j99ph -n kube-system
2018/10/24 15:19:05 [INFO] Starting: Version: v1.7.0, BuildInfo: n1260-443736128

BIG-IP Version

admin@(localhost)(cfg-sync Standalone)(Active)(/Common)(tmos)# show sys version

Sys::Version
Main Package
Product BIG-IP
Version 13.1.1
Build 0.0.4
Edition Final
Date Fri Jul 20 17:55:49 PDT 2018

Helm / Tiller Version

f5user@k8s-master:~$ helm version
Client: &version.Version{SemVer:"v2.11.0", GitCommit:"2e55dbe1fdb5fdb96b75ff144a339489417b146b", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.11.0", GitCommit:"2e55dbe1fdb5fdb96b75ff144a339489417b146b", GitTreeState:"clean"}

Diagnostic Information

Note the "unknown flag" error in the logs:

f5user@k8s-master:~$ kubectl logs exegetical-markhor-f5-bigip-ctlr-84bc995db6-x5x4z -n kube-system
unknown flag: --partition
Usage of /app/bin/k8s-bigip-ctlr
  Global:
      --http-listen-address string   Optional, address to serve http based informations (/metrics and /health). (default "0.0.0.0:8080")
      --log-level string             Optional, logging level (default "INFO")
      --node-poll-interval int       Optional, interval (in seconds) at which to poll for cluster nodes. (default 30)
      --python-basedir string        DEPRECATED: Optional, directory location of python utilities
      --verify-interval int          Optional, interval (in seconds) at which to verify the BIG-IP configuration. (default 30)
      --version                      Optional, print version and exit.

  BigIP:
      --bigip-partition stringArray    Required, partition(s) for the Big-IP kubernetes objects.
      --bigip-password string          Required, password for the Big-IP user account.
      --bigip-url string               Required, URL for the Big-IP
      --bigip-username string          Required, user name for the Big-IP user account.
      --credentials-directory string   Optional, directory that contains the BIG-IP username, password, and/or url files. To be used instead of username, password, and/or url arguments.

  Kubernetes:

----snip----

add notifications to repo

Charts repo should have notifications to slack similar to other repos.

See https://github.com/F5Networks/k8s-bigip-ctlr/blob/79df67c58504fc3f93139a38dfbafa3a7a665bae/.travis.yml#L74

[Feature Request] Add support for multiple networks (send the public IP and not the internal k8s ip)

What chart am I using:
kubernetes bigip management (v2)
What version of Bigip am I using:
12.1.1
What version of the chart/images am I using
1.5.x
What is the feature you want
We need a way to provide the correct IP on the correct network for the BIGIP to be configured properly.

Right now the F5 kubernetes integration sends the internal cluster IP to the bigip to configure the nodes. This does not work if we work with Calico/Canal and add multiple networks (example: 1 internal net, 1 public net)