Zipkin traces not showing up

[istio]

Zipkin traces not showing up

[debianmaster]

turns out this is a date issue on the container. when docker is running container host time is not synced into running container.

[debianmaster]

sidecar should send metrics to istio control plane in istio-system , AFAIK

[ldemailly]

can you add the relevant logs ? anything in mixer or envoy logs that seems abnormal ?

[debianmaster]

no sign of any errors in mixer or envoy logs i can see metadata but not any traces now. i changed date ranges and tried too

[debianmaster]

is there any dependency on kube 1.6 for distributed tracing to work?

[ldemailly]

also, are you running 0.1.6 or something else ?

[debianmaster]

i'm running 0.1.6 istio on openshift 1.5.1 which is k8s 1.5 :)

[ldemailly]

(edit: showing how to run latest pilot and istioctl / newer than 0.1.6:)

$ git clone https://github.com/istio/istio.git ; cd istio
$ source istio.VERSION 
$ export MANAGER_HUB=$PILOT_HUB  # Important/needed at the moment
$ export MANAGER_TAG=$PILOT_TAG  # ditto
$ curl -L $ISTIOCTL_URL/istioctl-osx > ./istioctl  # change the -osx to your env if needed
$ chmod 755 ./istioctl
$ ./istioctl version
istioctl version:

Version: 0.2.0-ac94a44
GitRevision: ac94a44
GitBranch: master
User: jenkins@ubuntu-16-04-build-12744189088e0f
GolangVersion: go1.8
KubeInjectHub: docker.io/istio
KubeInjectTag: 0.1


apiserver version:

Version: 0.2.0-ac94a44
GitRevision: ac94a44
GitBranch: master
User: jenkins@ubuntu-16-04-build-12744189088e0f
GolangVersion: go1.8.1
# rest of the installation instructions as is

[Morriz]

So what steps can we take to fix this? I am running minikube on osx

[Morriz]

I found this one liner:
minikube ssh -- docker run -i --rm --privileged --pid=host debian nsenter -t 1 -m -u -n -i date -u $(date -u +%m%d%H%M%Y)

here: kubernetes/minikube#1378

[Morriz]

And this one goes with it:
minikube ssh -- sudo ln -sf /usr/share/zoneinfo/Europe/Amsterdam /etc/localtime

But I still get no trace ;|

[Morriz]

Any idea what could be wrong? The date settings are now identical between host and guest vm.

I am using auto-injection.

[debianmaster]

@Morriz this error is 0.1.6 i do not see this error anymore on the newer alpha release

[Morriz]

I am using 0.2.4, but I get no metrics in zipkin nor jaeger

[debianmaster]

jaeger has some issue AFAIk, zipkin is working for me on 0.2.4 release

[ldemailly]

try master or wait a bit for 0.2.6 which should be today

[Morriz]

I deployed the book app in the default namespace (as in the tutorial). How is it going to deliver metrics to the istio-system namespace where I deployed the zipkin and jaeger services (again, following the tutorial)? The sidecars are running next to the pods in the 'default` namespace, no?

Maybe this is a documentation/intructions issue?

[Morriz]

Maybe it has to do with this: istio/istio#936

[Morriz]

I managed to get output...it just took forever ;| Starting zipkin again after jaeger got zipkin working, but jaeger seems to discard input after a small window of receiving metrics