Comments (11)
I've done some research and I understand better the origin of these errors, with helm template not using the namespace from the context for some obscure reasons that seem to be a great source of debate in the community (for instance helm/helm#3553).
But after some additional investigations, I found out that Helmfile already implements a "secret feature" to get around this issue, which is the forceNamespace parameter, to be defined for each release. This parameter makes Helmfile complete the manifests generated by helmfile template with the namespace from the configuration (see roboll/helmfile#326 (comment) π).
So our errors have been fixed, thank you for your reactivity, we'll keep using your great plugin, and come back to you if we face any further issue π
from argo-cd-helmfile.
Wow great research and references! Thanks for sharing your findings.
from argo-cd-helmfile.
Does this help that? https://argo-cd.readthedocs.io/en/stable/user-guide/sync-options/#create-namespace
from argo-cd-helmfile.
If any one was looking at this and still needed ideas. I tried using the HELMFILE_INIT_SCRIPT_FILE
, just created a basic bash script. It can create namespaces on remote clusters or just locally in the cluster. I use environment variables so this determines which releases you want to target and grabs the namespace to create and the hooks to run on each release.
It is a little ugly, but it runs quickly, and efficiently.
create-ns.sh
#!/bin/bash
function KUBE_CONF {
KUBE_CONFIG_PATH="/tmp/kube.yaml"
# Get config
SECRETCONFIGJSON=$(kubectl get secret "server-${ENVIRONMENT}" -n argocd -o jsonpath='{.data.config}' | base64 --decode | jq .)
CLIENTCERT=$(echo ${SECRETCONFIGJSON} | jq '.tlsClientConfig.certData')
CLIENTKEY=$(echo ${SECRETCONFIGJSON} | jq '.tlsClientConfig.keyData')
CA=$(echo ${SECRETCONFIGJSON} | jq '.tlsClientConfig.caData')
# Get Server
SECRETSERVER=$(kubectl get secret "server-${ENVIRONMENT}" -n argocd -o jsonpath='{.data.server}' | base64 --decode)
CONFIG="---
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: ${CA}
server: ${SECRETSERVER}
name: default
contexts:
- context:
cluster: default
user: default
name: default
current-context: default
kind: Config
preferences: {}
users:
- name: default
user:
client-certificate-data: ${CLIENTCERT}
client-key-data: ${CLIENTKEY}
"
# Create temporary kubeconfig
echo "$CONFIG" | yq eval-all '.' > /tmp/kube.yaml
export KUBECONFIG="${KUBE_CONFIG_PATH}"
}
function RUN_HELM_HOOKS {
HELMFILE_LOCATION="${PWD}/helmfile.yaml"
# Start helmfile hooks
HELMFILE_RELEASES=$(helmfile -f ${HELMFILE_LOCATION} list ${HELMFILE_GLOBAL_OPTIONS} --output json | jq -r '.[] | select(.enabled) | "\(.name)=\(.namespace)"')
for r in $HELMFILE_RELEASES; do
NAME=$(echo "$r" | grep -oE "^[^=]+")
NS=$(echo "$r" | grep -oE "[^=]+$")
# Check NS
kubectl get ns "$NS" &>/dev/null
if [[ $? -ne 0 ]]; then
kubectl create ns "$NS"
else
continue
fi
# Check Hooks
HELMFILE_HOOKS=$(rname="$NAME" yq eval '.releases[] | select(.name == env(rname)) | .hooks.[].command' ${HELMFILE_LOCATION})
if [ -z "$HELMFILE_HOOKS" ]; then
continue
fi
for h in $HELMFILE_HOOKS; do
h=$(basename "$h")
if [ $h == "Your hook name 1" ]; then
# Run your hook
fi
if [ $h == "Your hook name 2" ]; then
# Run the second hook
fi
done
done
}
function CLEAN_UP {
unset KUBECONFIG
rm -f $KUBE_CONFIG_PATH
}
# Get the environment based on what you set "-e" to
ENVIRONMENT=$(echo "$HELMFILE_GLOBAL_OPTIONS" | grep -oP '\-e \K.*')
# IF it is a local deployment in the cluster, only run helm hooks. The pod knows the cluster config
if [[ "${ENVIRONMENT}" == *"Your local environment variable"* ]]; then
RUN_HELM_HOOKS
else
KUBE_CONF
RUN_HELM_HOOKS
fi
CLEAN_UP
Just mount this in the helmfile plugin pod, and set the parameters in the app:
- name: HELMFILE_INIT_SCRIPT_FILE
value: '/tmp/scripts/create-ns.sh'
from argo-cd-helmfile.
Try setting the env HELMFILE_USE_CONTEXT_NAMESPACE=true
and see what you get. The readme has a few details and/or look at the src to see how it changes the output.
from argo-cd-helmfile.
From what I understand, it's only useful if the namespace is set in the Application, to kinda ignore it and use the context instead, but that's not even our case. We tried setting this variable nonetheless, but it didn't change anything.
I guess we'll have to debug the script manually to find where the issue could come from π
from argo-cd-helmfile.
Ok, given that weβre templating everything entirely and then argo applies the values in a single operation it posses a challenge (to do something like post-processing) because the context is lost at that point.
I think helmfile has an ability to do a post render hook on a per release basis where hypothetically you could find all namespaced assets and try to inject the ns if not present.
from argo-cd-helmfile.
Hello,
Thank you for sharing issue with namespaces - really helped me solving the same issue this issue creator stumbled on.
But after that I do have another issue:
When I want to create / sync for the first time, namespaces which are provided in helmfile
namespace: test
forceNamespace: test
are not being created and argo-cd throws an error that "Sync Failed", because "Nnamespaces "test" not found"
Am I missing some additional flag or env added to deployment? I am currently using only HELMFILE_GLOBAL_OPTIONS="-e test" as it is environment based deployment.
Thank you.
from argo-cd-helmfile.
Yes, that helps.
Thank you!
So just to be clear.. It is not enough to simply specify namespaces in helmfile and additional stuff (as mentioned in your answer) needs to be deployed. Can this be added to helmfile deployment or has to be done separately, I mean before deployment?
I should note that my helmfile contains multiple namespaces that needs to be created.
from argo-cd-helmfile.
That's another issue we also faced after using the forceNamespace parameter of Helmfile, and we were not able to fix it with the CreateNamespace option of Argo.
https://argo-cd.readthedocs.io/en/stable/user-guide/sync-options/#create-namespace
Note that the namespace to be created must be informed in the spec.destination.namespace field of the Application resource.
As we have a multi-namespaces Application, we shouldn't set the spec.destination.namespace
field, and so the CreateNamespace option is not working.
We didn't work more on this yet, but I guess we'll have to craft something as a workaround, like preInstall Helmfile hooks to create the namespaces, but if anyone have a better idea I'll take it π
from argo-cd-helmfile.
What have I did to go as a workaround:
{{- if .Values.namespaces.test.enabled }}
apiVersion: v1
kind: Namespace
metadata:
name: {{ .Values.namespaces.test.name }}
{{- end }}
and then added in values files values/keys respectively. Not the cleanest workaround, but it works.
Although, another issue appears. For example my full helmfile deployment contains prometheus-stack deployment and for some reason argo-CD does not install CRDs, even if I set in prometheus values file installCRDs: true
from argo-cd-helmfile.
Related Issues (20)
- Helmfile is moved HOT 2
- Error: unknown flag: --skip-crds HOT 10
- Passing args to helm HOT 11
- Issues related to sidecar CMP mode HOT 13
- Autodiscover with apps in app HOT 10
- Helm repository not found HOT 13
- adding kustomize transformers: causes breakage HOT 3
- helmfile/vals integration HOT 7
- Not allow helm valuesFiles path or Values file as block file with plugin helmfile
- Create Application per Helmfile Release HOT 1
- argo-cd-helmfile.sh discover` failed exit status 1 HOT 1
- Fail to fetch OCI private Helm registry HOT 1
- Integrated ArgoCD plugin parameters support ? HOT 3
- Setting --skip-deps as optional HOT 2
- Not installing crds in folder of helm chart HOT 2
- What about ExternalSecrets instead of sops? HOT 1
- Not able to use secretref+k8s to fetch secrets from Kubernetes Secret object HOT 2
- kubectl context support HOT 3
- Pass helm values from separate repo HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from argo-cd-helmfile.