[Feature] Support Cilium - add parsing of conntrack information

[fstr]

[Feature] Support Cilium - add parsing of conntrack information

[fstr]

@Nils98Ar correct, only pods that use hostNetwork (e. g. cilium-agent) will be tracked, which causes mis-attribution. All other pod traffic won't be tracked at all.

[chipzoller]

Thanks for the enhancement request here. It seems worthwhile to add specific support for Cilium, perhaps via a container flag which can be configured via a Helm value. The current version of network costs doesn't actually use conntrack any longer and instead requires on sys calls to gather the traffic flows.

Instead of using conntrack tools, the conntrack information can be listed using cilium bpf ct list global in the Cilium agent, which runs in privilged mode as a DaemonSet on every node. As far as I know, the information is stored on the host in /sys/fs/bpf/tc/globals/cilium_ct4_global.

So when it comes to adding support for Cilium in network costs, would its Pods be able to simply parse this file or would they need to possibly exec into Cilium's agent Pods as well (or in place of)? Trying to understand the level of effort required to support Cilium here.

[fstr]

Hi @chipzoller, thanks for your response.

As far as I know, it is possible to programmatically read the ebpf information from any program running on the same host as the Cilium agent. I know that this was done by other people already. An example can be found in the linked project in my original post.

While it's possible to "exec" into the Cilium agent pod, I'd say you want to gather the information directly from the host system.

[Nils98Ar]

@fstr So the network cost feature does not currently work correctly if cilium kubeProxyReplacement is enabled?

[Nils98Ar]

@fstr @chipzoller Using the Kubernetes node's traffic flow information also does not work with the Cilium default routing mode encapsulation/tunnel (vxlan/geneve) regardless of wether you run Cilium kube-proxy-less or not. I think Ciliums conntrack information needs to be used here as well.

When pod p1 on node 1 communicates with a pod p2 on node 2 you only see a connection from pod p1 IP to node 2 IP ("remote node") in the traffic flows (at least in the conntrack table) on node 1. You do not see the IP pod p2 there.

So you cannot use the traffic classification by destination (internet, in-zone, in-region, etc.) for Pod-IPs within the Cluster. E.g. we wanted to add the Ingress Controllers Pod IPs to the internet destinations.