Unable to run Kubevela in Kubernetes v1.29

[adityasinghal26]

Unable to run Kubevela in Kubernetes v1.29

[lucasafonsokremer]

Adding further,

I have installed a new Kubernetes cluster in AWS and setup kubevela installation using Helm with vela v1.9.9. Apparently everything is fine for Kubernetes v1.29.1 em vela 1.9.9.

KubeVela Version

CLI Version: 1.9.9
Core Version: 1.9.9

% kubectl version
Client Version: v1.28.3
Server Version: v1.29.1-eks-b9c9ed7

% kubectl get pods -n vela-system
NAME                                         READY   STATUS    RESTARTS   AGE
kubevela-cluster-gateway-0o47f78c64-64st8   1/1     Running   0          4m11s
kubevela-vela-core-as9x87bff-98xo9         1/1     Running   0          4m11s

Additional context

I didn't visualize errors on vela-core and i was able to create an kubevela application, running a nginx for tests

[FogDong]

This is caused by the bug of v1.9.10.
I created a PR for fixing the client bug for v1.9.10, once it's merged, we'll release v1.9.11 and it should be fixed.

[dhiguero]

Thanks @adityasinghal26 , @xqm32 for reporting the issue. I will try to reproduce and investigate the root cause.

[dhiguero]

I was able to reproduce the issue performing the installation using vela install on Kubernetes v1.26.0, v1.26.14, v1.27.11, v1.28.7, and v1.29.2. And it seems that the issue may be on the vela CLI itself as using the helm approach on v1.29.2 seems to be working. I will suggest trying this approach while the issue is investigated on vela:

helm repo add kubevela https://charts.kubevela.net/core
helm repo update
helm install --create-namespace -n vela-system kubevela kubevela/vela-core --wait

$ k apply -f basic_app.yaml
application.core.oam.dev/first-vela-app created

$ k get pods
NAME                              READY   STATUS    RESTARTS   AGE
express-server-565ccffd86-lg2qh   1/1     Running   0          9s

$ k get app
NAME             COMPONENT        TYPE         PHASE     HEALTHY   STATUS      AGE
first-vela-app   express-server   webservice   running   true      Ready:1/1   12s

[xqm32]

Same error here.

kubectl version

Client Version: v1.29.3
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.2

vela version

CLI Version: 1.9.10
Core Version: 1.9.10
GitRevision: 24756fc507665bff5cbd6108e63391a3e50fa96b
GolangVersion: go1.22.1

[manjunathmkotabal]

Kubevela officially support Kubernetes cluster >= v1.19 && <= v1.26. that's why it is not working properly in 1.29

[adityasinghal26]

Hi @dhiguero,

I tried the above approach by using Helm earlier. The vela-core app was failing with the same error. I tried to install kubevela again using Helm commands that you mentioned above (adding below) in a fresh namepsace velaa. I am currently using Kubernetes v1.29.1 (installed using Docker Desktop).

helm repo add kubevelaa https://charts.kubevela.net/core
helm repo update
helm install --create-namespace -n velaa kubevelaa kubevela/vela-core --wait

The vela-core pods is in Running state as in the screenshot above. However, the pod logs represent a different scenario with same errors again.

I0413 09:58:07.297014       1 componentdefinition_controller.go:64] "Reconcile componentDefinition" componentDefinition="velaa/worker"
E0413 09:58:07.297537       1 revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"
I0413 09:58:09.065582       1 policydefinition_controller.go:62] "Reconciling PolicyDefinition..." Name="apply-once" Namespace="velaa"
I0413 09:58:09.065750       1 policydefinition_controller.go:62] "Reconciling PolicyDefinition..." Name="envbinding" Namespace="velaa"
I0413 09:58:09.065950       1 policydefinition_controller.go:62] "Reconciling PolicyDefinition..." Name="garbage-collect" Namespace="velaa"
I0413 09:58:09.067557       1 policydefinition_controller.go:62] "Reconciling PolicyDefinition..." Name="override" Namespace="velaa"
E0413 09:58:09.075152       1 revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"
E0413 09:58:09.076257       1 revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"
E0413 09:58:09.076729       1 revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"
E0413 09:58:09.077156       1 revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"
0413 09:58:09.078926       1 controller.go:329]  "msg"="Reconciler error" "error"="the server could not find the requested resource (patch policydefinitions.core.oam.dev override)" "PolicyDefinition"={"name":"override","namespace":"velaa"} "controller"="policydefinition" "controllerGroup"="core.oam.dev" "controllerKind"="PolicyDefinition" "name"="override" "namespace"="velaa" "reconcileID"="4f902650-f898-421d-82ec-ffadc416d267"
I0413 09:58:09.079030       1 policydefinition_controller.go:62] "Reconciling PolicyDefinition..." Name="read-only" Namespace="velaa"
E0413 09:58:09.080864       1 revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"

Just to test the application resource, I also create a basic app using https://kubevela.io/docs/quick-start/#deploy-a-classic-application. I am able to see the application created but not able to see anything in the pod list.

% kubectl apply -f basic_app.yaml
application.core.oam.dev/first-vela-app created
% kubectl get velaapp --all-namespaces
NAMESPACE   NAME             COMPONENT        TYPE         PHASE   HEALTHY   STATUS   AGE
default     first-vela-app   express-server   webservice                              2m2s
% kubectl get pods
No resources found in default namespace.

On inspecting further, below are the details when describing the velaapp. It shows no events for the app. I believe, we might need to upgrade kubevela for the latest Kubernetes versions.

% kubectl describe velaapp first-vela-app
Name:         first-vela-app
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  core.oam.dev/v1beta1
Kind:         Application
Metadata:
  Creation Timestamp:  2024-04-13T10:02:47Z
  Generation:          1
  Resource Version:    1315418
  UID:                 e5149fa4-204a-4ff7-9c88-bca3a55ed35c
Spec:
  Components:
    Name:  express-server
    Properties:
      Image:  oamdev/hello-world
      Ports:
        Expose:  true
        Port:    8000
    Traits:
      Properties:
        Replicas:  1
      Type:        scaler
    Type:          webservice
  Policies:
    Name:  target-default
    Properties:
      Clusters:
        local
      Namespace:  default
    Type:         topology
    Name:         target-prod
    Properties:
      Clusters:
        local
      Namespace:  prod
    Type:         topology
    Name:         deploy-ha
    Properties:
      Components:
        Traits:
          Properties:
            Replicas:  2
          Type:        scaler
        Type:          webservice
    Type:              override
  Workflow:
    Steps:
      Name:  deploy2default
      Properties:
        Policies:
          target-default
      Type:  deploy
      Name:  manual-approval
      Type:  suspend
      Name:  deploy2prod
      Properties:
        Policies:
          target-prod
          deploy-ha
      Type:  deploy
Events:      <none>

[adityasinghal26]

Adding further,

I have installed a new Kubernetes cluster in local using kind utility and setup vela installation using Helm commands as in last comment. The error persists for Kubernetes v1.29.2.

kind create cluster --image=kindest/node:v1.29.2 --name=kind29-2

% kubectl version
Client Version: v1.29.3
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.2
% kubectl -n velaa get pods
NAME                                         READY   STATUS    RESTARTS   AGE
kubevelaa-cluster-gateway-5bb6f78c64-82rh7   1/1     Running   0          9m18s
kubevelaa-vela-core-5f88c87bff-qtfrq         1/1     Running   0          9m18s

E0413 10:45:53.716150       1 revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"
E0413 10:45:53.717433       1 controller.go:329]  "msg"="Reconciler error" "error"="the server could not find the requested resource (patch workflowstepdefinitions.core.oam.dev list-config)" "WorkflowStepDefinition"={"name":"list-config","namespace":"velaa"} "controller"="workflowstepdefinition" "controllerGroup"="core.oam.dev" "controllerKind"="WorkflowStepDefinition" "name"="list-config" "namespace"="velaa" "reconcileID"="df5e26cc-846c-4fd6-be3d-faee6c852024"
E0413 10:45:53.721594       1 controller.go:329]  "msg"="Reconciler error" "error"="the server could not find the requested resource (patch traitdefinitions.core.oam.dev resource)" "TraitDefinition"={"name":"resource","namespace":"velaa"} "controller"="traitdefinition" "controllerGroup"="core.oam.dev" "controllerKind"="TraitDefinition" "name"="resource" "namespace"="velaa" "reconcileID"="1d99f3a8-8b9d-4855-8a8f-1ef426a1f2dc"
I0413 10:45:53.811156       1 workflowstepdefinition_controller.go:65] "Reconciling WorkflowStepDefinition..." Name="clean-jobs" Namespace="velaa"
E0413 10:45:53.811872       1 revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"
E0413 10:45:53.813681       1 controller.go:329]  "msg"="Reconciler error" "error"="the server could not find the requested resource (patch workflowstepdefinitions.core.oam.dev clean-jobs)" "WorkflowStepDefinition"={"name":"clean-jobs","namespace":"velaa"} "controller"="workflowstepdefinition" "controllerGroup"="core.oam.dev" "controllerKind"="WorkflowStepDefinition" "name"="clean-jobs" "namespace"="velaa" "reconcileID"="62eae767-2015-45a9-bb23-ad4b538d9ea9"
I0413 10:45:53.912814       1 workflowstepdefinition_controller.go:65] "Reconciling WorkflowStepDefinition..." Name="build-push-image" Namespace="velaa"

[Sachin5411]

facing same issue for our setup as well :

% kubectl version
Client Version: v1.28.3
Server Version: v1.28.6-eks-508b6b3

helm list -n vela-system
NAME    	NAMESPACE  	REVISION	UPDATED                            	STATUS  	CHART           	APP VERSION
kubevela	vela-system	1       	2024-04-15 21:57:27.85141 +0530 IST	deployed	vela-core-1.9.10	1.9.10

E0415 16:31:42.806599       1 controller.go:329]  "msg"="Reconciler error" "error"="the server could not find the requested resource (patch traitdefinitions.core.oam.dev ingress)" "TraitDefinition"={"name":"ingress","namespace":"vela-system"} "controller"="traitdefinition" "controllerGroup"="core.oam.dev" "controllerKind"="TraitDefinition" "name"="ingress" "namespace"="vela-system" "reconcileID"="05210090-00a4-4961-9cd3-3d770ab73a2c"
I0415 16:31:42.902780       1 traitdefinition_controller.go:64] "Reconcile traitDefinition" traitDefinition="vela-system/scaler"
I0415 16:31:42.903218       1 workflowstepdefinition_controller.go:65] "Reconciling WorkflowStepDefinition..." Name="apply-component" Namespace="vela-system"
E0415 16:31:42.904018       1 revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"
E0415 16:31:42.904324       1 revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"
E0415 16:31:42.907039       1 controller.go:329]  "msg"="Reconciler error" "error"="the server could not find the requested resource (patch traitdefinitions.core.oam.dev scaler)" "TraitDefinition"={"name":"scaler","namespace":"vela-system"} "controller"="traitdefinition" "controllerGroup"="core.oam.dev" "controllerKind"="TraitDefinition" "name"="scaler" "namespace"="vela-system" "reconcileID"="e24d5a21-77d6-4625-832d-ac08a75f0490"
E0415 16:31:42.907214       1 controller.go:329]  "msg"="Reconciler error" "error"="the server could not find the requested resource (patch workflowstepdefinitions.core.oam.dev apply-component)" "WorkflowStepDefinition"={"name":"apply-component","namespace":"vela-system"} "controller"="workflowstepdefinition" "controllerGroup"="core.oam.dev" "controllerKind"="WorkflowStepDefinition" "name"="apply-component" "namespace"="vela-system" "reconcileID"="af342392-8810-4376-9b4e-dbc6f2ad4368"