Kubernetes cluster with 100+ service namespaces

[istio]

Kubernetes cluster with 100+ service namespaces

[rshriram]

We've fixed this issue and will be making a release soon.

On Wed, Jul 26, 2017 at 2:44 AM suresh ***@***.***> wrote: +1 same need for us too. We have namespaces based on team/stack they manage (eg: frontend, backend) and all service from frontend talks to backend. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#16 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AH0qd0zEDDzmouSX6nPwY_SMvIbamy-Iks5sRuBWgaJpZM4NnRxb> .

-- ~shriram

[rshriram]

Hi, we have fixed this issue in 0.2.4. You can now run your applications in any namespace, while running Istio control plane in istio-system namespace. Please test it out and let us know if it works for your use case.

[JeanMertz]

Ah wait, I forgot I read something about this yesterday:

For the 0.1 release, Istio must be installed in the same Kubernetes namespace as the applications.

So I guess that answers my own question? Istio currently only supports a single namespace, correct?

[andraxylia]

For this release, Istio must be run in the same namespace as the application. You can deploy Istio multiple times in separate namespaces, in the same cluster, but this will not help in your cluster. This is because we configure istio-manager and other components to listen to k8s events in a particular namespace.
For the next release, we plan to support a cluster-wide Istio deployment, where Istio-manager, Istio-Mixer, Istio-Ingress, Istio-Egress and Istio-CA are deployed once per cluster, in a namespace called istio-system, and serves applications/services deployed in their own namespaces. This model will better fit your cluster, I hope.

Your use-case is interesting. May I ask what was the reason for separating each service in its own namespace?

[andraxylia]

istio/istio#338

[andraxylia]

Do you need all services to be visible and accessible to each-other, forming a complete mesh, or you would like to enforce other policies?

[JeanMertz]

Your use-case is interesting. May I ask what was the reason for separating each service in its own cluster?

(I'm going to assume you meant namespace, not cluster)

Each service lives in its own namespace, with its own secrets, configmaps, services, etc. This is a nice encapsulation, and allows us to:

1. be sure that any resources related to a service are scrubbed/deleted if we delete that specific namespace
2. allow devs/teams to quickly set the context to a specific namespace
3. have our company-wide CI easily know where to deploy services without any special configuration (we use the <github-repo-name>-<environment> naming scheme)
4. allow using a tool like stern to easily log all pods of a service
5. let's us automate things like listing all services (the application kind, not Kubernetes kind) (ie. all namespaces) in our cluster, and thus at our company, we can then add extra metadata and/or annotations to automate things like responsible teams/people for projects, contact email addresses, etc.

All in all, it neatly encapsulates the concept of a "service" and let's us automate our entire workflow around it.

Do you need all services to be visible and accessible to each-other, forming a complete mesh, or you would like to enforce other policies?

What other policies are you thinking of? I think adding extra visibility/discovery separation between namespaces would be a nice bonus, but having all services be visible to each other would definitely suit our initial needs 👍

[kuroneko25]

Same use case for us as well. We have a multi-tenant GKE cluster and we assign different namespaces to each tenant to manage resources that way.

[FuzzOli87]

I would also imagine this would be necessary for clusters with "qa/staging/production" namespaces and be able to work an "integration testing" layer by temporarily routing traffic across namespaces.

[bjartek]

We have the same need. We deploy several apps to a namespace, but each affiliation in the organization have their own namespaces/projects (we are on openshift). We also have several clusters because of legacy network infrastructure.

[sureshvis]

+1 same need for us too. We have namespaces based on team/stack they manage (eg: frontend, backend) and all service from frontend talks to backend.

[andraxylia]

Not yet ready, but unblocked, and working actively on it.

[emedina]

Hi, I'm using 0.2.4, but this seems not to be working properly yet.

I have defined an Ingress rule like this:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: istio
  name: ag-helper-ingress
  namespace: ag-helper
spec:
  rules:
  - host: aghelper.arch.oami.eu
    http:
      paths:
      - backend:
          serviceName: ag-source-helper-service
          servicePort: 80
        path: /source.*
      - backend:
          serviceName: ag-processor-helper-service
          servicePort: 80
        path: /processor.*
      - backend:
          serviceName: ag-sink-helper-service
          servicePort: 80
        path: /sink.*
      - backend:
          serviceName: ag-ui-helper-service
          servicePort: 80
        path: /ui.*

Double-checked that all services belong in the ag-helper namespace, but I always get a 404.

On the other side, the echoheaders sample in the default namespace works fine.

[andraxylia]

Does East-West traffic work?

[eigokor]

if you will check your Ingress configuration with

istioctl  get ingress-rule ag-helper-ingress-1-1

you will notice that uri type regex which seems to be not supported.

try to remove ".*" in your configuration. It will change type to "prefix" which is supported.

I ve asked this question, but no luck to have any response yet.
https://stackoverflow.com/questions/46399248/istio-incorrect-handle-of-ingress-backend-path

[emedina]

Istio 0.2.4 does not have ingress-rule option, only routerules.

As mentioned in my post before, the echoheaders in the default namespace works like a charm using path: /headers.*, so such regex seems not to be the issue here, but the different namespace.

Can anyone confirm?

Created issue --> https://github.com/istio/issues/issues/83

[emedina]

After some more research, I realized that the problem was actually on the definition of my services which were missing a port name: http.

Now it works like a charm.

Thanks everyone for your help.

[andraxylia]

Glad to see it works.

[andraxylia]

Fixed in 0.2.6 by multiple PRs tracked by epic istio/istio#338.