tiswanso/examples

Network Service Mesh examples repo

examples's Issues

nsmgr pod in crashloopback state

This is related to issue at: networkservicemesh/networkservicemesh#1972
Tried interdomain feature as per demo at NSMCon, nsmgr is in crashloopback state
nsm-system nsmgr-chcct 2/3 CrashLoopBackOff 208 18h

logs of nsmd-k8s container looks like:

time="2019-12-11T14:24:08Z" level=info msg="Version: 75502fff"
time="2019-12-11T14:24:08Z" level=info msg="Creating logger from config: &{nsmd-k8s@nsmgr-chcct false false [] 0xc000388000 0xc00010e120 <nil> <nil> <nil>}"
2019/12/11 14:24:08 Initializing logging reporter
time="2019-12-11T14:24:08Z" level=info msg="Starting NSMD Kubernetes on 0.0.0.0:5000 with NsmName gke-vitta1-default-pool-7302dc57-81jg"
time="2019-12-11T14:24:08Z" level=info msg="GRPC.NewServer with open tracing enabled"
time="2019-12-11T14:24:08Z" level=info msg="Replacing Network services with: [{{NetworkService networkservicemesh.io/v1alpha1} {vl3-service  default /apis/networkservicemesh.io/v1alpha1/namespaces/default/networkservices/vl3-service 38162033-1b71-11ea-a92e-42010a800217 77050 1 2019-12-10 17:19:27 +0000 UTC <nil> <nil> map[] map[] [] nil []  []} {IP []} {}}]"
time="2019-12-11T14:24:08Z" level=info msg="Replacing Network service endpoints with: [{{NetworkServiceEndpoint networkservicemesh.io/v1alpha1} {vl3-service7vjlx vl3-service default /apis/networkservicemesh.io/v1alpha1/namespaces/default/networkserviceendpoints/vl3-service7vjlx 381bb7c9-1b71-11ea-a92e-42010a800217 77051 1 2019-12-10 17:19:27 +0000 UTC <nil> <nil> map[app:vl3-nse-ucnf networkservicename:vl3-service] map[] [] nil []  []} {vl3-service IP gke-vitta1-default-pool-7302dc57-81jg} {RUNNING}}]"
time="2019-12-11T14:24:08Z" level=info msg="Add from k8s-registry: *v1alpha1.NetworkService"
time="2019-12-11T14:24:08Z" level=info msg="Add from k8s-registry: *v1alpha1.NetworkServiceEndpoint"
time="2019-12-11T14:24:08Z" level=info msg="Replacing Network service endpoints with: [{{NetworkServiceManager networkservicemesh.io/v1alpha1} {gke-vitta1-default-pool-7302dc57-81jg  default /apis/networkservicemesh.io/v1alpha1/namespaces/default/networkservicemanagers/gke-vitta1-default-pool-7302dc57-81jg fa2c4f36-1b70-11ea-a92e-42010a800217 76692 1 2019-12-10 17:17:43 +0000 UTC <nil> <nil> map[] map[] [] nil []  []} {} {2019-12-10 17:17:43 +0000 UTC 10.44.0.10:30501 RUNNING}}]"
time="2019-12-11T14:24:08Z" level=info msg="NetworkServiceManagerCache.Added(&{{NetworkServiceManager networkservicemesh.io/v1alpha1} {gke-vitta1-default-pool-7302dc57-81jg  default /apis/networkservicemesh.io/v1alpha1/namespaces/default/networkservicemanagers/gke-vitta1-default-pool-7302dc57-81jg fa2c4f36-1b70-11ea-a92e-42010a800217 76692 1 2019-12-10 17:17:43 +0000 UTC <nil> <nil> map[] map[] [] nil []  []} {} {2019-12-10 17:17:43 +0000 UTC 10.44.0.10:30501 RUNNING}})"
time="2019-12-11T14:24:08Z" level=info msg="RegistryCache started"
time="2019-12-11T14:24:08Z" level=info msg="nsmd-k8s initialized and waiting for connection"
time="2019-12-11T14:24:08Z" level=error msg="configmaps \"kubeadm-config\" not found"
time="2019-12-11T14:24:08Z" level=info msg="Start monitoring prefixes to exclude"
time="2019-12-11T14:24:08Z" level=info msg="GRPC.NewServer with open tracing enabled"
time="2019-12-11T14:24:08Z" level=info msg="Waiting for liveness probe: unix:/var/lib/networkservicemesh/plugins/registry.sock"
time="2019-12-11T14:24:08Z" level=info msg="Start apply filter by namespace default for &{{NetworkServiceManager networkservicemesh.io/v1alpha1} {gke-vitta1-default-pool-7302dc57-81jg  default /apis/networkservicemesh.io/v1alpha1/namespaces/default/networkservicemanagers/gke-vitta1-default-pool-7302dc57-81jg fa2c4f36-1b70-11ea-a92e-42010a800217 76692 1 2019-12-10 17:17:43 +0000 UTC <nil> <nil> map[] map[] [] nil []  []} {} {2019-12-10 17:17:43 +0000 UTC 10.44.0.10:30501 RUNNING}}"
time="2019-12-11T14:24:08Z" level=info msg="Add from k8s-registry: *v1alpha1.NetworkServiceManager"
time="2019-12-11T14:24:08Z" level=info msg="NetworkServiceManagerCache.Added(&{{NetworkServiceManager networkservicemesh.io/v1alpha1} {gke-vitta1-default-pool-7302dc57-81jg  default /apis/networkservicemesh.io/v1alpha1/namespaces/default/networkservicemanagers/gke-vitta1-default-pool-7302dc57-81jg fa2c4f36-1b70-11ea-a92e-42010a800217 76692 1 2019-12-10 17:17:43 +0000 UTC <nil> <nil> map[] map[] [] nil []  []} {} {2019-12-10 17:17:43 +0000 UTC 10.44.0.10:30501 RUNNING}})"
2019/12/11 14:24:08 Reporting span 68eecd9a07b3fa91:68eecd9a07b3fa91:0:1
time="2019-12-11T14:24:08Z" level=fatal msg="Failed to start K8s Plugin rpc error: code = Unknown desc = already have a plugin with the same name"

Steps performed:

1. Created single node cluster using GKE and EKS consoles
2. Obtained kubeconfig of above clusters
3. Followed step1 and 2 of https://github.com/tiswanso/examples/tree/demo_vl3/examples/vl3_basic (so checked out tiswanso/networkservicemesh and tiswanso/examples of branches specified in README.md)
4. And then the steps at Mysql Demo example in same page

At this point, mysql installations failed probably due to variables in helm.. but, a container in nsmgr also in crashloopback state and its logs are as shown above.