openzipkin/zipkin

Zipkin is a distributed tracing system

zipkin's Issues

Document implementing a tracer

We should add a section to the readme describing how to implement a tracer for a protocol/library. Finagle and Thrift is probably the easiest example to describe.

Publish to Maven Central

I discussed with @franklinhu on creating a Sonatype OSS account:
https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide

This would allow us to publish artifacts directly to Maven Central and be good open source citizens.

Download links for most release files appear to be broken

Attempting to download most of the files on the releases page results in a "File not found" message.

Deploy script

Right now deploying the open source version of Zipkin is a very manual process. We need to reduce that burden a bit. Capistrano script would probably do as a start.

getTraceIdsByAnnotation

test in CassandraIndexSpec, why is

      // fetch by time based annotation, find trace
      var seq = cassandraIndex.getTraceIdsByAnnotation("service", "custom", None, 0, 3)()
      //seq mustEqual Seq(span1.traceId)

commented out? It seems like it isn't time based (the duration hasn't been indexed) but also like it should be found regardless.

also, what does the test:

      // should not find any traces since the core annotation doesn't exist in index
      seq = cassandraIndex.getTraceIdsByAnnotation("service", "cs", None, 0, 3)()
      seq.isEmpty mustBe true

mean? Why shouldn't the core annotation exist in index, and what does core annotation mean?

I'm implementing this for redis, and it seems like the reverse should be true, that the second should be commented out, and the first should be uncommented.

collector drops traces with certain ids

Reported instances of the collector dropping specific trace IDs.

XXX: By experimentation zipkin has trouble recording traces with ids
larger than (2 ** 56) - 1

https://github.com/racker/tryfer/blob/master/tryfer/trace.py#L26

Copy link to this search button

Imagine you've just searched for something and you want to share that results page. Similar to how Google maps has a link called something like "link to this map" we could have a "link to this search". It would create a link that recreates the search for another user.

Update docs with protocol debug flag and annotation duration

web: better client/server differentiation

Would be nice to be able to clearly differentiate between the parts of the span

There should be an optional lower time bound for fetching traces

is support finagle 6.3 & scala 2.10?

my demo project under finagle 6.3 & scala 2.10, i try connect zipkin but get IncompatibleClassChangeError

here is my code

class AskServiceConfig extends ServerConfig[AskService.ThriftServer] {

 var tracerFactory: Tracer.Factory = ZipkinTracer.apply(scribeHost = "127.0.0.1", scribePort = 9410, sampleRate = 1.0f )

FAT [20130415-09:27:00.076] Ask: A server service  threw an exception
FAT [20130415-09:27:00.076] Ask: java.lang.IncompatibleClassChangeError: Implementing class
FAT [20130415-09:27:00.076] Ask:     at java.lang.ClassLoader.defineClass1(Native Method)
FAT [20130415-09:27:00.076] Ask:     at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
FAT [20130415-09:27:00.076] Ask:     at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
FAT [20130415-09:27:00.076] Ask:     at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
FAT [20130415-09:27:00.076] Ask:     at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
FAT [20130415-09:27:00.076] Ask:     at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
FAT [20130415-09:27:00.076] Ask:     at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
FAT [20130415-09:27:00.076] Ask:     at java.security.AccessController.doPrivileged(Native Method)
FAT [20130415-09:27:00.076] Ask:     at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
FAT [20130415-09:27:00.076] Ask:     at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
FAT [20130415-09:27:00.076] Ask:     at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.zipkin.thrift.scribe$log_args.write(scribe.scala:203)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.scrooge.FinagleThriftClient$$anonfun$encodeRequest$1.apply(FinagleThriftClient.scala:27)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.scrooge.FinagleThriftClient$$anonfun$encodeRequest$1.apply(FinagleThriftClient.scala:22)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.util.Try$.apply(Try.scala:13)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.util.Future$.apply(Future.scala:56)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.scrooge.FinagleThriftClient$class.encodeRequest(FinagleThriftClient.scala:22)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.zipkin.thrift.scribe$FinagledClient.encodeRequest(scribe.scala:290)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.zipkin.thrift.scribe$FinagledClient.log(scribe.scala:305)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.zipkin.thrift.RawZipkinTracer.logSpan(RawZipkinTracer.scala:123)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.zipkin.thrift.RawZipkinTracer.mutate(RawZipkinTracer.scala:145)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.zipkin.thrift.RawZipkinTracer.annotate(RawZipkinTracer.scala:217)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.zipkin.thrift.RawZipkinTracer.record(RawZipkinTracer.scala:156)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.zipkin.thrift.ZipkinTracer.record(ZipkinTracer.scala:79)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.tracing.Trace$$anonfun$record$1.apply(Trace.scala:192)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.tracing.Trace$$anonfun$record$1.apply(Trace.scala:192)
FAT [20130415-09:27:00.076] Ask:     at scala.collection.immutable.Set$Set1.foreach(Set.scala:86)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.tracing.Trace$.record(Trace.scala:192)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.tracing.Trace$.record(Trace.scala:224)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.thrift.ThriftServerTracingFilter$$anonfun$apply$2$$anonfun$apply$3.apply(ThriftServerFramedCodec.scala:173)
FAT [20130415-09:27:00.076] Ask:     (...more...)

search for traces by a service's duration

Would be nice to be able to sort traces by the searched service's duration and not the duration of the entire trace.

Merge script fails

I attempted to merge a pull request, but it seems our little merge scripts is broken somehow.

[marburg zipkin (master)]$ bin/git-pull-request.rb merge 245
Executing: 'git config --get github.token'
warning: peer certificate won't be verified in this SSL session
Executing: 'git config --get github.token'
warning: peer certificate won't be verified in this SSL session
bin/git-pull-request.rb:187:in `merge': Not merging into master (RuntimeError)
from bin/git-pull-request.rb:306

bin/sbt script throws an error

When running the bin/sbt script I get this error.
bin/sbt: line 16: [: too many arguments

It works as expected though, but we should fix the problem anyway.

scala 2.9.2

Zipkin isn't on 2.9.2, do you mind if I update it to 2.9.2?

Improve scribe documentation

Documentation on getting the full stack up and running should be more thorough.

collector: move sample/storagerequest rate thrift methods to admin interface

The collector thrift interface has a few admin methods to get and set the sample rate and storage request rate. These should be moved into the ostrich admin interface.

Scala 2.10 Artifacts

Are there artifacts that support Scala 2.10 ?

http://maven.twttr.com/ has just artifacts for the 2.9 release.
There also aren't any on maven central.

What to do to get 'em?

Try using Travis CI Again

I think it should be a goal of the project to be running on some type of CI system.

We should try to get things running on Travis CI again if possible.

Simplify configuration

We've had quite a bit of feedback via the mailing lists that the configurations of the collector, query service, and web UI are quite confusing. Users are unclear which parts they do and do not need, and are often put off by there being too many moving parts. In reality, the basic collector/query setup only requires a backing store (either Cassandra or Redis at the time of this writing).

The base configs are pretty confusing, so I'm proposing moving to a Finagle Builder style config.
Base config:

CollectorBuilder(Scribe.Interface)
  .writeTo(Cassandra.static("hostname"))
  .writeTo(Cassandra.zookeeperServerSet("/zookeeper/path/to/cassandra/cluster/serverset"))

Or maybe you want to use Redis instead

CollectorBuilder(...)
  .writeTo(Redis.static(...))

Or maybe you want to listen for JSON on an HTTP port (doesn't exist yet, but is conceivable)

CollectorBuilder(Http.Json)

Or, if you want something more intense with sampling

CollectorBuilder(...)
  .sampler(Sampler.static(0.5)) // sample 50% of spans

Or even adaptive sampling

CollectorBuilder()
  .sampler(AdaptiveSampler.zookeeper(...))

The builders would be similar for the query service and the web UI. This would also allow for users to spin up all three services in the same JVM instance by returning a Seq[Builder]

Thoughts?

Search broken on traces page?

Is it just me or is the search bar completely unusable? No queries are returning any results.

Also, I wonder if ctrl + F works for your guys? For me, In Firefox, ctrl + F doesn't highlight the text being searched, whereas in Chrome ctrl + F just doesn't work on the traces page.

ui: Buffer underflow

FAT [20121023-18:33:11.993] ZipkinWeb: java.nio.BufferUnderflowException
FAT [20121023-18:33:11.993] ZipkinWeb:     at java.nio.Buffer.nextGetIndex(Buffer.java:498)
FAT [20121023-18:33:11.993] ZipkinWeb:     at java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:355)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.zipkin.conversions.json$WrappedBinaryAnnotation.toJson(json.scala:19)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.zipkin.conversions.json$WrappedTraceTimeline$$anonfun$toJson$10.apply(json.scala:79)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.zipkin.conversions.json$WrappedTraceTimeline$$anonfun$toJson$10.apply(json.scala:79)
FAT [20121023-18:33:11.993] ZipkinWeb:     at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:194)
FAT [20121023-18:33:11.993] ZipkinWeb:     at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:194)
FAT [20121023-18:33:11.993] ZipkinWeb:     at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:60)
FAT [20121023-18:33:11.993] ZipkinWeb:     at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:44)
FAT [20121023-18:33:11.993] ZipkinWeb:     at scala.collection.TraversableLike$class.map(TraversableLike.scala:194)
FAT [20121023-18:33:11.993] ZipkinWeb:     at scala.collection.mutable.ArrayBuffer.map(ArrayBuffer.scala:44)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.zipkin.conversions.json$WrappedTraceTimeline.toJson(json.scala:79)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.zipkin.conversions.json$WrappedTraceCombo$$anonfun$toJson$12.apply(json.scala:95)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.zipkin.conversions.json$WrappedTraceCombo$$anonfun$toJson$12.apply(json.scala:95)
FAT [20121023-18:33:11.993] ZipkinWeb:     at scala.Option.map(Option.scala:133)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.zipkin.conversions.json$WrappedTraceCombo.toJson(json.scala:95)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.zipkin.web.App$$anonfun$12$$anonfun$apply$23.apply(App.scala:236)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.zipkin.web.App$$anonfun$12$$anonfun$apply$23.apply(App.scala:235)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Future$$anonfun$map$1$$anonfun$apply$10.apply(Future.scala:503)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Try$.apply(Try.scala:13)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Future$.apply(Future.scala:98)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Future$$anonfun$map$1.apply(Future.scala:503)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Future$$anonfun$map$1.apply(Future.scala:503)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Future$$anonfun$flatMap$1.apply(Future.scala:477)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Future$$anonfun$flatMap$1.apply(Future.scala:476)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Promise$$anonfun$transform$1.apply(Future.scala:883)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Promise$$anonfun$transform$1.apply(Future.scala:879)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Promise$$anonfun$respondWithoutChaining$1.apply(Future.scala:868)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Promise$$anonfun$respondWithoutChaining$1.apply(Future.scala:863)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.concurrent.IVar$$anonfun$runqs$1.apply$mcV$sp(IVar.scala:174)
FAT [20121023-18:33:11.993] ZipkinWeb:     (...more...)

Parallelizing tests?

The build and tests takes quite awhile for Zipkin...

Can we speed things up by parallelizing?

http://about.travis-ci.org/blog/2012-11-28-speeding-up-your-tests-by-parallelizing-them/

Simplify documentation for first time users

• Set Finagle sampleRate to 1
• Set collector sample rate to 1
• Make collector sample rate a top level config option

How can I quickly try out a zipkin demo?

I just wanna try out a zipkin demo to check whether it is suitable for my own project. I follow the Install page , and I have already built the zipkin project successfully by using the command bin/sbt update package-dist, it shows no errors and all tests are passed. So, what's the next, I can not find a dist directory to execute scp dist/zipkin*.zip [server].

I just wanna see a demo to demonstrate the functionality of zipkin. How can I try out the Zipkin UI(maybe the zipkin-web directory, I am not sure) to see the rails app?

Maybe the Install Page is out of date?

sbt update failed

I tried to insall zipkin, when run

bin/sbt update package-dist

got the error info below,

[info] Resolving commons-httpclient#commons-httpclient;3.1 ...
[info] Resolving commons-logging#commons-logging;1.0.4 ...
[info] Resolving commons-codec#commons-codec;1.2 ...
[info] Resolving org.scala-tools.sbt#completion_2.9.1;0.11.2 ...
[info] Resolving jline#jline;0.9.94 ...
[info] Resolving org.scala-tools.sbt#run_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#task-system_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#tasks_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#tracking_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#cache_2.9.1;0.11.2 ...
[info] Loading project definition from /home/qinyuchun/source/zipkin-setup/zipkin/project
[info] Updating {file:/home/qinyuchun/source/zipkin-setup/zipkin/project/}default-64d38a...
[info] Resolving com.twitter#sbt-package-dist;1.0.5 ...
[info] Resolving ivysvn#ivysvn;2.1.0 ...
[info] Resolving org.markdownj#markdownj;0.3.0-1.0.2b4 ...
[info] Resolving org.freemarker#freemarker;2.3.16 ...
[info] Resolving com.twitter#sbt11-scrooge;3.0.0 ...
[info] Resolving com.twitter#sbt-thrift2;0.0.1 ...
[info] Resolving com.eed3si9n#sbt-assembly;0.8.2 ...
[info] Resolving org.scala-tools.sbt#sbt_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#main_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#actions_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#classfile_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#io_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#control_2.9.1;0.11.2 ...
[info] Resolving org.scala-lang#scala-library;2.9.1 ...
[info] Resolving org.scala-tools.sbt#interface;0.11.2 ...
[info] Resolving org.scala-tools.sbt#logging_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#process_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#classpath_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#launcher-interface_2.9.1;0.11.2 ...
[info] Resolving org.scala-lang#scala-compiler;2.9.1 ...
[info] Resolving org.scala-tools.sbt#incremental-compiler_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#collections_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#api_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#persist_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbinary#sbinary_2.9.0;0.4.0 ...
[info] Resolving org.scala-tools.sbt#compile_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#ivy_2.9.1;0.11.2 ...
[info] Resolving org.apache.ivy#ivy;2.2.0 ...
[info] Resolving com.jcraft#jsch;0.1.31 ...
[info] Resolving org.scala-tools.sbt#testing_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.testing#test-interface;0.5 ...
[info] Resolving org.scala-tools.sbt#compiler-interface;0.11.2 ...
[info] Resolving org.scala-tools.sbt#precompiled-2_8_1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#precompiled-2_8_0;0.11.2 ...
[info] Resolving org.scala-tools.sbt#precompiled-2_9_0;0.11.2 ...
[info] downloading http://maven.twttr.com/com/twitter/sbt-thrift2_2.9.1_0.11.2/0.0.1/sbt-thrift2-0.0.1.jar ...
[warn] [FAILED ] com.twitter#sbt-thrift2;0.0.1!sbt-thrift2.jar: Downloaded file size doesn't match expected Content Length for http://maven.twttr.com/com/twitter/sbt-thrift2_2.9.1_0.11.2/0.0.1/sbt-thrift2-0.0.1.jar. Please retry. (52886ms)
[warn] [FAILED ] com.twitter#sbt-thrift2;0.0.1!sbt-thrift2.jar: Downloaded file size doesn't match expected Content Length for http://maven.twttr.com/com/twitter/sbt-thrift2_2.9.1_0.11.2/0.0.1/sbt-thrift2-0.0.1.jar. Please retry. (52886ms)
[warn] ==== twitter.com: tried
[warn] http://maven.twttr.com/com/twitter/sbt-thrift2_2.9.1_0.11.2/0.0.1/sbt-thrift2-0.0.1.jar
[info] downloading http://maven.twttr.com/ivysvn/ivysvn/2.1.0/ivysvn-2.1.0.jar ...
[warn] [FAILED ] ivysvn#ivysvn;2.1.0!ivysvn.jar: Downloaded file size doesn't match expected Content Length for http://maven.twttr.com/ivysvn/ivysvn/2.1.0/ivysvn-2.1.0.jar. Please retry. (54377ms)
[warn] [FAILED ] ivysvn#ivysvn;2.1.0!ivysvn.jar: Downloaded file size doesn't match expected Content Length for http://maven.twttr.com/ivysvn/ivysvn/2.1.0/ivysvn-2.1.0.jar. Please retry. (54377ms)
[warn] ==== twitter.com: tried
[warn] http://maven.twttr.com/ivysvn/ivysvn/2.1.0/ivysvn-2.1.0.jar
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: FAILED DOWNLOADS ::
[warn] :: ^ see resolution messages for details ^ ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: ivysvn#ivysvn;2.1.0!ivysvn.jar
[warn] :: com.twitter#sbt-thrift2;0.0.1!sbt-thrift2.jar
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[error] {file:/home/qinyuchun/source/zipkin-setup/zipkin/project/}default-64d38a/*:update: sbt.ResolveException: download failed: ivysvn#ivysvn;2.1.0!ivysvn.jar
[error] download failed: com.twitter#sbt-thrift2;0.0.1!sbt-thrift2.jar
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore?

Why I start zipkin webUI, it throws a Unknown method exception

When I start web UI and query the trace log, it throws a Unknown method exception
Unsafe is a class in rt.jar. But I try all JDK such as JDk 7u17,openJDK7,JDK6u22,openJDK6 , I still cannot resolve this problem.
I wonder this sun.misc.Unsafe.copyMemory(Ljava/lang/Object;JLjava/lang/Object;JJ)V In which JDK version .

FAT [20130402-13:28:03.663] ZipkinWeb: org.apache.thrift.TApplicationException: Internal error processing getTraceSummariesByIds: 'java.lang.NoSuchMethodError: sun.misc.Unsafe.copyMemory(Ljava/lang/Object;JLjava/lang/Object;JJ)V'
FAT [20130402-13:28:03.663] ZipkinWeb: at org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.zipkin.gen.ZipkinQuery$FinagledClient.decodeResponse(ZipkinQuery.scala:5263)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.zipkin.gen.ZipkinQuery$FinagledClient$$anonfun$getTraceSummariesByIds$1.apply(ZipkinQuery.scala:5480)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.zipkin.gen.ZipkinQuery$FinagledClient$$anonfun$getTraceSummariesByIds$1.apply(ZipkinQuery.scala:5479)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.util.Future$$anonfun$flatMap$1.apply(Future.scala:477)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.util.Future$$anonfun$flatMap$1.apply(Future.scala:476)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.util.Promise$$anonfun$transform$1.apply(Future.scala:883)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.util.Promise$$anonfun$transform$1.apply(Future.scala:879)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.util.Promise$$anonfun$respondWithoutChaining$1.apply(Future.scala:868)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.util.Promise$$anonfun$respondWithoutChaining$1.apply(Future.scala:863)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.concurrent.IVar$$anonfun$runqs$1.apply$mcV$sp(IVar.scala:174)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.concurrent.IVar$LocalScheduler.run(IVar.scala:125)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.concurrent.IVar$LocalScheduler.apply(IVar.scala:105)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.concurrent.IVar.runqs(IVar.scala:169)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.concurrent.IVar.set(IVar.scala:311)

ZipkinSpec doesn't tear down zookeeper instance

When running tests for zipkin server, the ZipkinSpec doesn't close the ZK instance that's started. This causes another run of the tests to fail since the ZK instance is still bound to the port.

To reproduce:

  $ bin/sbt
  > test // this may pass
  > ...
  > test // ZipkinSpec will fail

Start scripts

We don't have any easy way of starting up the open source version. Need to add some scripts for that.

traces aren't sorted when displayed in chrome

chrome has an unstable sort, so although spans are sorted by start_time when the traces/get_trace.json endpoint gets called, they become mixed up when they're sorted again.

bin/sbt update package-dist fails to resolve elephant-bird-cascading dependency

[info] Resolving com.twitter.elephantbird#elephant-bird-cascading2;3.0.0 ...
[warn] module not found: com.twitter.elephantbird#elephant-bird-cascading2;3.0.0
[warn] ==== ibiblio: tried
[warn] http://mirrors.ibiblio.org/pub/mirrors/maven2/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== twitter.com: tried
[warn] http://maven.twttr.com/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== powermock-api: tried
[warn] http://powermock.googlecode.com/svn/repo/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== scala-tools.org: tried
[warn] http://scala-tools.org/repo-releases/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== testing.scala-tools.org: tried
[warn] http://scala-tools.org/repo-releases/testing/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== oauth.net: tried
[warn] http://oauth.googlecode.com/svn/code/maven/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== download.java.net: tried
[warn] http://download.java.net/maven/2/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== atlassian: tried
[warn] https://m2proxy.atlassian.com/repository/public/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== jboss: tried
[warn] http://repository.jboss.org/nexus/content/groups/public/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== local-lookup: tried
[warn] file:/Users/arul/.m2/repository/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== local: tried
[warn] ==== elephant-bird repo: tried
[warn] http://oss.sonatype.org/content/repositories/comtwitter-105/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== Concurrent Maven Repo: tried
[warn] http://conjars.org/repo/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[info] Resolving org.scala-tools.testing#specs_2.9.1;1.6.9 ...
[info] Resolving org.scala-lang#scala-compiler;2.9.1 ...
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: com.twitter.elephantbird#elephant-bird-cascading2;3.0.0: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[error] {file:/Users/arul/twitter/zipkin/}zipkin-hadoop/*:update: sbt.ResolveException: unresolved dependency: com.twitter.elephantbird#elephant-bird-cascading2;3.0.0: not found

Is this dependency publicly available?

Get links of traces

Is there a way to get the URL to a specific trace? So, having a trace id, how do I programmatically get the URL to the web page of that trace?

Thanks,
Derek

CassandraAggregates Futures

As Johan mentions, you shouldn't have to block on the remove in the storage methods, but instead chain the futures so you can return one that executes this first and then the batch.

Cleanup modules

Currently we have a confusing hierarchy of modules for the collector and query services including:

• zipkin-common: common classes shared between the two
• zipkin-thrift: Thrift structs and interfaces for both services
• zipkin-scrooge: module to generate Scala from Thrift via Scrooge
• zipkin-scribe: scribe specific classes for collector, including the service
• zipkin-server: combination of code for collector and query, including the query service

I think we can clean this up by breaking up the latter two modules and make things more composable.

• zipkin-cassie: implementation of Aggregates, Index, and Storage interfaces that use Cassandra as the backing store
• zipkin-collector-core: core code for the collector service
• zipkin-collector-scribe: scribe specific classes
• zipkin-collector-service: module to compose everything together
• zipkin-query-core: core code for query service
• zipkin-query-service: module to compose everything together

Upgrade to Finatra 1.1.1

1.1.1 has a bunch of goodies over 0.2.4 and is available on Maven Central.

Cleanup scald.rb

The way to launch the hadoop jobs is terrifying. We should make this easier/more intuitive
https://github.com/twitter/zipkin/blob/master/zipkin-hadoop/src/scripts/scald.rb

Zipkin Test links broken

Shows a file not found error.

Better hadoop job documentation

Should include steps to run preprocessors

sbt default repository is blocked in China, any other building solutions?

Sbt's default repository http://repo.typesafe.com/typesafe/ivy-releases/ is blocked by GFW in China. When I try to execute the command 'sbt', it keeps downloading the jars needed.

So, is there any other repository that sbt can use? Or, any other building solutions?

Use Future[Unit], not Future[Void] in interfaces

Errors thrown by InetAddress.getLocalHost

When running the web server locally, I encountered the following error:

[info] Running com.twitter.zipkin.web.Main -f zipkin-web/config/web-dev.scala
Nov 24, 2012 12:56:02 AM com.twitter.logging.Logger log
INFO: Loading configuration
Nov 24, 2012 12:56:02 AM java.util.logging.LogManager$RootLogger log
FATAL: Error in config file: %s
java.net.UnknownHostException: birderator: birderator
    at java.net.InetAddress.getLocalHost(InetAddress.java:1438)
    at com.twitter.zipkin.config.ZipkinConfig$class.$init$(ZipkinConfig.scala:30)
    at Evaluator__web$2ddev_16de0e15bca3885a651af7857b6a8434a7dbff86$$anon$1.<init>((inline):23)
    at Evaluator__web$2ddev_16de0e15bca3885a651af7857b6a8434a7dbff86.apply((inline):23)
    at Evaluator__web$2ddev_16de0e15bca3885a651af7857b6a8434a7dbff86.apply((inline):1)
    at com.twitter.util.Eval.applyProcessed(Eval.scala:197)
    at com.twitter.util.Eval.apply(Eval.scala:167)
    at com.twitter.ostrich.admin.RuntimeEnvironment.loadConfig(RuntimeEnvironment.scala:230)
    at com.twitter.ostrich.admin.RuntimeEnvironment.loadRuntimeConfig(RuntimeEnvironment.scala:256)
    at com.twitter.zipkin.web.Main$.main(Main.scala:28)
    at com.twitter.zipkin.web.Main.main(Main.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at sbt.Run.invokeMain(Run.scala:66)
    at sbt.Run.run0(Run.scala:59)
    at sbt.Run.execute$1(Run.scala:48)
    at sbt.Run$$anonfun$run$1.apply$mcV$sp(Run.scala:52)
    at sbt.TrapExit$.executeMain$1(TrapExit.scala:33)
    at sbt.TrapExit$$anon$1.run(TrapExit.scala:42)
Caused by: java.net.UnknownHostException: birderator
    at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
    at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:866)
    at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1258)
    at java.net.InetAddress.getLocalHost(InetAddress.java:1434)
    ... 20 more

This is because InetAddress.getLocalHost is not equivalent to InetAddress.getHostByName("localhost"). I have a small branch that resolves this issue by using InetAddress.getHostByName("localhost")

Sort query results according to selected sort order

When doing a search, the query results are always rendered with the Duration Desc sort order rather than the selected one.

How to Install Zipkin on Windows7?

Hi,

Can you please guide me how to install Zipkin on Windows7 OS and how to use it.

Thanks,
Krishna

Prefix or regex lookups for annotations

We've received numerous requests for prefix lookups or even regex matching for annotations when looking up traces. This would allow users to match a prefix to a memcache key for example.

Query by multiple key/value pairs

Sometimes it's useful to be able to search for multiple key/value pairs. For example in order to find traces with 500 errors for a particular url.

key=http.uri, value=/i/discovery.json
key=http.responsecode, value=500 Internal Server Error

Display url in the lookup results

On the trace lookup page we can display the url of each trace in the list to make it easier to pick interesting ones. We would have to make the kv annotations that represent the url configurable as different instrumented libraries might call them differently.

Why I cannot use aggregate in zipkin-web

When I click aggregate in zipkin-web,the page will always in running and cannot open the result. And I haven`t seen any Exception in console.
I wonder How can I use the aggregate function.

Counter logs successful send despite failure when SLF4J logger missing

When there's no SLF4J logger implementation on the classpath, the Zipkin log_span counter logs a successful send when it's actually failing due to a NoClassDefFoundException. Using a simple console StatsReceiver with no logger implementation, the output is as follows:

Counter HttpServer:connects = 1
Counter HttpServer:received_bytes = 149
Counter HttpServer:requests = 1
Stat HttpServer:handletime_us = 61161.0
Stat HttpServer:request_latency_ms = 4.0
Counter HttpServer:success = 1
Counter zipkin:create_log_entries:error:java.lang.NoClassDefFoundError: org/slf4j/impl/StaticLoggerBinder = 1
Counter HttpServer:sent_bytes = 19
Stat HttpServer:connection_received_bytes = 149.0
Stat HttpServer:connection_sent_bytes = 19.0
Stat HttpServer:connection_duration = 170.0
Stat HttpServer:connection_requests = 1.0
Counter zipkin:log_span:ok = 1

... but the span is not actually sent to Scribe.

zipkin-web: download trace link is broken

The sample rate maybe wrong

The sample rate is 2, 4, 8, 16 ... 1024 in google Dapper, but in the finagle source, I find the sample value is a float value and can be set by any rate. I thought that may lose some feature in zipkin.