Issue with proxy (Ingress-Nginx) and externalTrafficPolicy to preserve source IP

[aerott]

Issue with proxy (Ingress-Nginx) and externalTrafficPolicy to preserve source IP

[aerott]

I think the enableServicesElection flag is equal svc_election (please look here: #455 (comment) ).

My Ingress-Nginx (reverse proxy) configuration is:

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx \
  --create-namespace \
  --namespace ingress-basic \
  --set controller.replicaCount=2 \
  --set controller.service.type=LoadBalancer \
  --set controller.service.loadBalancerIP=170.18.1.121 \
  --set controller.service.externalTrafficPolicy=Local

[aerott]

@Cellebyte - Thank you for your FYI. As soon as the new image (version) is available, I will try to do a test.

[reefland]

Trying to solve the same problem with Traefik. No luck yet. I get the IP of the Traefik POD not the client IP.

Did you review Kube-VIP External Traffic Policy where it says to add:

- name: enableServicesElection
    value: "true"

I didn't see that in your provided configuration, and set externalTrafficPolicy: Local on the LB?

[gielvandanu]

I have the same problem. If I deploy the ingress nginx using helm with default mode (using externalTrafficPolicy=Cluster) it gets the IP for the services. But if I add the --set controller.service.externalTrafficPolicy=Local flag when deploy, pending,

I'm using ghcr.io/kube-vip/kube-vip:v0.6.3

But the service describe shows it gives the IP address :

kubectl -n ingress-nginx describe service/ingress-nginx-controller
Name:                     ingress-nginx-controller
Namespace:                ingress-nginx
Labels:                   app.kubernetes.io/component=controller
                          app.kubernetes.io/instance=ingress-nginx
                          app.kubernetes.io/managed-by=Helm
                          app.kubernetes.io/name=ingress-nginx
                          app.kubernetes.io/part-of=ingress-nginx
                          app.kubernetes.io/version=1.9.4
                          helm.sh/chart=ingress-nginx-4.8.3
                          implementation=kube-vip
Annotations:              kube-vip.io/loadbalancerIPs: 10.66.250.16
                          meta.helm.sh/release-name: ingress-nginx
                          meta.helm.sh/release-namespace: ingress-nginx
Selector:                 app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
Type:                     LoadBalancer
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.100.50.132
IPs:                      10.100.50.132
IP:                       10.66.250.16
Port:                     http  80/TCP
TargetPort:               http/TCP
NodePort:                 http  31148/TCP
Endpoints:                10.244.4.24:80
Port:                     https  443/TCP
TargetPort:               https/TCP
NodePort:                 https  32675/TCP
Endpoints:                10.244.4.24:443
Session Affinity:         None
External Traffic Policy:  Local
HealthCheck NodePort:     30869
Events:
  Type    Reason                Age                From                Message
  ----    ------                ----               ----                -------
  Normal  EnsuringLoadBalancer  61s (x2 over 62s)  service-controller  Ensuring load balancer
  Normal  EnsuredLoadBalancer   61s (x2 over 61s)  service-controller  Ensured load balancer
  Normal  LoadbalancerIP        61s                service-controller  -> 10.66.250.16`

[d-uzlov]

Could this be related to node names, like in this issue?

• #608

Kube-vip advertises Local services from nodes where relevant pods are present.
When k8s node name didn't match hostname output, kube-vip couldn't check it, so it never advertised the assigned address, and never updated the external IP status of the service.

Maybe fix for that issue broke something, or maybe it wasn't fully fixed.
I currently don't have any test clusters, so I can't check.

[aerott]

@d-uzlov
I'm not sure if I'm understanding the #608 issue related to hostname well, but I think it is a little different case.

My node name:

My hostname:

[sematula]

Having the same issue. When externalTrafficPolicy is set to cluster, no problems. When set to local, services get stuck in pending. Tried with 5.11 and 6.4. My node names and hostnames are identical, and no fqdn is set. This is a local bare metal cloud built on Ubuntu 22.04 and microk8s.

[Cellebyte]

FYI. We got a small fix merged which fixes the shortName lookup for hosts which have no fqdn set.

• hostname.example.com was working
• hostname was not working.

Fix was -> #724 and my issue was #723

[cyberops7]

Chiming in here to say that I'm seeing the same/similar issue, running version 0.6.4. After updating the kube-vip manifest with

- name: svc_election
  value: "true"

All sorts of things broke, including loadBalancer services stuck with External-IP pending. As soon as I removed those lines from the manifest, the kube-vip-ds pods redeployed and everything came back online.

I saw this error frequently in the kube-vip-ds-* pod logs while it was in the broken state:

time="2024-01-30T06:27:26Z" level=error msg="[endpoint] unable to find shortname from k3s-m2"

My three kube-vip-ds pods are running on my 3 master nodes (k3s-m{1..3}), and each pod had the same errors logged with the respective shortname.

Related question: when trying to use this feature (my desire here is to preserve source IP), does it matter that the kube-vip-ds pods are only running on the master nodes? Or should they be running on all nodes?

[Cellebyte]

@cyberops7 @aerott again, can you verify on your side if the latest release helps with your discovered issues?

[aerott]

@cyberops7 @aerott again, can you verify on your side if the latest release helps with your discovered issues?

Unfortunately, it's still the same. I can't connect externally to Ingress Nginx, even though there are no visible errors anywhere.

Update note:
My K8s cluster is currently running with:

• Kubernetes Version: v.1.29.1 (kubeadm; 3 CP nodes)
• Kube-vip Version: 0.7.0