Plugin fails to connect with deny ingress NetworkPolicy

[pierluigilenoci]

Plugin fails to connect with deny ingress NetworkPolicy

[michaelmdresser]

Hi @pierluigilenoci. That's an interesting error to get, it looks like your flags are correct. It may be an issue with your network configuration in Kubernetes. Can you try the following, and let me know what happens? It may help us debug this problem.

kubectl proxy --port 8080

curl -G 'http://localhost:8080/api/v1/namespaces/k8s-kubecost/services/cost-analyzer-cost-analyzer:tcp-model/proxy/getConfigs'

kubectl cost uses the Kubernetes API server to proxy a request to Kubecost. If you have a restrictive firewalling or other network security policy, running this command may surface that.

[pierluigilenoci]

curl -G 'http://localhost:8080/api/v1/namespaces/k8s-kubecost/services/cost-
analyzer-cost-analyzer:tcp-model/proxy/getConfigs'
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {

  },
  "status": "Failure",
  "message": "error trying to reach service: dial tcp [REDACTED]:9003: i/o timeout",
  "code": 500
}

We have network policies in place. Ref: kubecost/cost-analyzer-helm-chart#743

[michaelmdresser]

Okay, not surprising if you have restrictive network policies in place. I'm unfamiliar with your setup and so cannot provide a recommendation. Allowing API servers to access the Kubecost API is a bit sketchy if you're worried about this kind of behavior in your cluster and have implemented policies preventing it.

This is a tricky situation to handle. The most robust, "behaves like kubectl" method I can think of is to manage a port-forward on the user's behalf while kubectl cost is running, but I can see many pitfalls there. As a short-term workaround, we could also modify kubectl cost only slightly to allow you to provide a URL to your Kubecost deployment to override this proxying behavior. Would the latter option satisfy your needs in the short-term?

[pierluigilenoci]

@michaelmdresser my setup is quite simple:

A deny all policy inside the Kubecost namespace:

---
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: deny-all-ingress-traffic
  namespace: k8s-kubecost
spec:
  podSelector: {}
  policyTypes:
    - Ingress

and this configuration for the Kubecost chart:

networkPolicy:
  enabled: true
  denyEgress: false
  sameNamespace: false
  namespace: k8s-kubecost

I believe that it is the basic setup of anyone who implements Network Policy: close everything and open only what is needed.

I would have expected that the networkPolicy option would have created something working for the kubectl plugin to work as well.

It's my opinion that finding workarounds to make everything work in an installation that should in fact be a standard is not a solution that satisfies me.

What do you think about it?

[michaelmdresser]

Naturally, a deny all ingress to the k8s-kubecost namespace will cause the current approach to fail. I understand what you want and how to address it. We can use client-go to port forward in code and maintain a session for the duration of the program. This is certainly a more robust approach. We will prioritize it, but of course contributions are welcome if you'd like to jump on it sooner @pierluigilenoci 😁.

References:

• in client-go: https://github.com/kubernetes/client-go/blob/master/tools/portforward/portforward.go
• blog post using this functionality: https://gianarb.it/blog/programmatically-kube-port-forward-in-go

[michaelmdresser]

Addressing in #80

[michaelmdresser]

With #80 merged and v0.2.0 released. I expect kubectl cost to work for you now. Please give it a try and let me know how it goes.

[michaelmdresser]

Sorry, we'll need to merge #82 and make a new release for this to work fully. I'll update you when that's done.

[michaelmdresser]

v0.2.1 is out and will hopefully have resolved this problem. Let me know how it goes.

[pierluigilenoci]

Works perfectly, thank you a lot! ❤️