Issues with current design

[cstockton]

Issues with current design

[cstockton]

@tedsuo Thanks a lot for this response, most of your points deserve taking some time to digest and think about. One thing that is unclear to me is the API/SDK separation, perhaps this is a terminology thing and I've failed to convey one of my biggest concerns or I'm failing to understand your roadmap. I have two choices in the current design:

1. Import a vendors binary blob into my code as a plugin and have that ship my traces
2. Use the "builtin" binary blob that ships to the OpenTelemetry agent via an API not suitable for widespread adoption as a neutral wire format which then imports all of the vendor binary blobs.

But I strongly feel that I should be able to import opentelemetry-go and ship traces anywhere using one vendor neutral wire format. Without that the community isn't truly open, the biggest players are just bundling their transports together. To be honest, I would have been happy with any first revision of a wire format. Comma separated streams of JSON- sounds good. Binary stream of gzipped excel spread sheets, sure, why not. At least then the SME's for kafka, rmq, insert cool tech here can begin writing agents, aggregators, queuing mechanisms and more cool tech here to fit all kinds of use cases and begin a strong feedback loop for this newly emerging wire format.

Most importantly our (non-contributors) hands are untied and we have something to build upon.

I respect and accept that my vision may not reflect yours. I also am in no place to continue to contest it (I'm not doing the work here!) - this was just a final attempt to convey it. I'm happy to agree to disagree and let everyone get back to work. Overall I'm thrilled and look forward to the projects progress.

[jmacd]

@cstockton I don't know if this helps your concern, but the prototype code that we merged in Go was very incomplete, and I've filed issue 20 about separating the API from the SDK properly, which is a requirement from OpenTracing. I am working on this separation.

In the terminology that's developing, an "exporter" is different than a "plugin". I think I agree with your reasoning. We can have an SDK that has practically no dependencies except standard libraries. An SDK could write to stdout or it could send spans to a server.

The OpenCensus style of library put a lot more logic into the client, which allows various kinds of exporter to be implemented in the process. I'm more interested in a client that quickly encodes data for another process to process and export data, which is why we have to separate the SDK and API here and in the other repos. Still, the goal is to end up like OpenTracing, where there's an API that you can use without knowing what the SDK provides. If you're objecting to the current state of the code, please wait for issue 20 to be resolved, after that and once the trace data format begins to crystalize, we can have an SDK that does very little other than encode bytes and write them somewhere.

[tigrannajaryan]

@cstockton I am not an OpenTelemetry Admin or a member of TSC, the following is just my personal opinion.

My understanding is that the primary current goal for OpenTelemetry is merging of OpenCensus and OpenTracing in backwards compatible manner. The motivation is to have a clear support and migration path for existing users of OpenCensus and OpenTracing. This is already a pretty difficult goal, anything on top of it should likely be avoided at this stage.

My opinion is that any improvements should be left for later. Attempting to merge these 2 projects and at the same time come up with significant new improvements and conceptual changes will make the project a lot more difficult and possibly result in a failure. I think that would be wrong and I support gradual change and improvement approach.

I understand your suggestion to design a better wire protocol and can see the value in the particular approach suggested (shipping span events instead of complete spans). I can also see downsides to the suggested approach of shipping events, so it is not so clear-cut (I can expand on this if interested).

I applaud your effort to come up with better solutions than what we already have, however I do not think process-wise what you propose helps with the project goals. I think a better process would be for you and other interested parties to work on your own proposal in parallel to current efforts by the rest of OpenTelemetry community, come up with design documents and sample implementation demonstrating the benefits, present it to OpenTelemetry community and aim to make it part of future OpenTelemetry API (V2).

I am personally of the opinion that existing wire protocols and memory representation of span data is not well-suited to certain high-performance use cases and I plan on following my own advice and building my case for a new high-perf proto that I will present to the community if my experiments are successful.

This brings me to the other related topic on which you commented - API compatibility - so I'll answer it here.

What I wrote above is precisely why I think it is important to have a well-defined API compatibility story and guarantees. I do not think that we will be able to come up with the "best" OpenTelemetry in the first version and that we will need to have a V2. If this is true then in my opinion it is very important that today we send a clear message to all OpenTelemetry adopters (especially to third-party library authors) that when V2 is introduced it will provide zero or almost zero-effort backward compatibility story.

Strong compatibility story de-risks OpenTelemetry adoption for third-party library authors and for end users and at the same time provides a path for radical improvements in the future (so that "can always change things later" is a technical possibility).

[iredelmeier]

@cstockton: without getting too the in weeds here - although I will add that I agree with many of your points! - any chance you'd mind filing an RFC (or two!) here?

[cstockton]

What I wrote above is precisely why I think it is important to have a well-defined API compatibility story and guarantees. I do not think that we will be able to come up with the "best" OpenTelemetry in the first version and that we will need to have a V2. If this is true then in my opinion it is very important that today we send a clear message to all OpenTelemetry adopters (especially to third-party library authors) that when V2 is introduced it will provide zero or almost zero-effort backward compatibility story.

This is where my fundamental disagreement is. Merging OpenTracing and OpenCensus is going to cause massive churn for the ecosystem already- more so on the OpenTracing end as it seems that project is essentially being deleted in favor of the OpenCensus design. I can not find any benefits to go through these efforts with plans to "change everything to v2 later" - we are forcing churn in 2 communities (OpenTracing OpenCensus -> OpenTelemetry) and again for OpenTelemetry V1 -> V2. Why not do our best to come out with a V1 that has a foundation we can build upon for years? Maybe it really is impossible, but I don't understand why we can't slow down a bit and try. Both ecosystems work today, what is with the sense of urgency here? :/

@iredelmeier - I've started an issue here open-telemetry/opentelemetry-specification#20 would changing this into a RFC in your repo yield a different outcome? Currently my issues have been dismissed under the premise "we can't change anything because of compatibility" while assuring me the same argument won't be valid in the future after N months of user adoption and thousands of lines of code are written. You'll have to forgive me if I'm skeptical about investing further time to voice my concerns to OpenTelemetry.

I'll leave it to the OpenTelemetry team to close my issues if they would like. I'll review code at some point for any security related concerns / major bugs, but refrain from commenting on the design decisions I fervently disagree with to keep progress moving forward for everyone. Thanks.

[bhs]

@cstockton thank you for raising this.

This issue/thread seems almost by-definition unresolvable given the number of meaty potential tangents above and the constraints of GitHub issues as a medium (maybe I'm arguing for the "threading" of an RFC PR as @iredelmeier has been advocating for). As such, I apologize in advance for only addressing a few of the points above... for the many aspects I'm not addressing, it's not because I think they're trivial (they are not).

All of that said, without further ado:

The sense of urgency: the current state (of OpenTracing, OpenCensus, and OpenTelemetry concurrently co-developing) is an unstable equilibrium for the larger ecosystem. This won't be resolved until there's a viable migration path from OpenTracing and OpenCensus code to OpenTelemetry. One blocker there could be backwards-incompatibility (which has been raised already in this thread). It's worth mentioning that an inadequate OpenTelemetry API could also be a blocker and should be taken seriously. So, the urgency is about getting to a stable equilibrium where OpenTelemetry is the clear best-choice of the three, as the unstable situation today will cause friction and anxiety; but the actual API (as opposed to the SDK) does require careful thought and scrutiny since poor execution there will be the ultimate counterweight to adoption.

Separation of API and SDK: I think @jmacd addressed this well already, but it's important to reiterate that this has been a high-level goal for the OpenTelemetry project since the beginning... e.g., the Loose coupling of components section of the blog post announcing the intention to merge these projects.

[tedsuo]

@cstockton on vacation currently, sorry for the rambling reply. To address your core concerns:

• The APIs are separated from SDKs, just like in OpenTracing. I really hope that is clear. I'm concerned that it isn't.
• There is a "default" SDK, based on the OpenCensus model, which is available to get people started. OpenTracing lacked a default, and this was a big issue for new users. But the SDK is not a required component. Nor is the default "agent/sidecar" currently being proposed. Nor is any particular wire protocol required (though, again, there is a default). All of these components must be mix and match, or it's in violation of the design of the project. If you see a hard dependency arise between these components, please raise an alarm.
• To prove there is complete API/SDK separation, I propose a litmus test: binding an event-based C++ implementation via foreign function calls. This is an excellent alternative implementation for those who require performance, works in almost all languages, and proves that the interfaces are separated from the SDK.
• The spec absolutely needs to be cleaned up. I have shared concerns here. We cannot deviate in design to the point that OpenTelemetry instrumentation cannot coexist with OpenTracing/OpenCensus instrumentation, and allow for a progressive migration of instrumentation. But we can make something cleaner than what we had in either projects. I will not allow (to the degree I am able) an OpenTelemetry v1.0 to ship while it is messy. There will be pressure to do so, given the deadlines we have set ourselves. But we will push the deadlines back, rather than ship something which is not finished baking. And we will reimplement the APIS in all the languages if we have to, several times, until we get it right. But I want those deadlines to push towards being really engaged in the spec writing this summer. There's a difference between spending time and letting time pass. Fwiw, the details of the current spec timeline is here. We must defer any additional functionality (such as a logs interface) while we focus on cleaning up tracing, metrics, and context propagation.
• There is a standard data format being proposed, along with semantic specifications for events. I agree that is an area where we need to front-load our attention. In fact, that's what I have listed on the timeline here. But obviously... it is not going to be done on july 6th as planned. I interpret this to mean our schedule is slipping, not that we are going to skip this step. Either an additional SIG focused on this, or additional meetings of the spec SIG, is needed in this area.
• But, also... there are a number of API concerns which are fairly independent of the data format. Obviously the API must be able to efficiently produce the data. But a good portion of the API design is driven by concerns which are not obvious from the perspective of a wire protocol - the data format alone will not save us. So I believe we still need to be getting our feet wet in the various languages, and getting a feedback cycle going between the language SIGs and the Spec SIG. I would love help making this process less messy!

I really hope this feels like "being heard" and not like everyone is telling you no... you raise good points, and also are showing places where the messaging is clearly not landing - we should not be having some of these convos in github issues; the answers should be clear and obvious. The main criticism that lands (for me, at least) is there are not nearly enough high level design requirements being written down, and the data format needs to be put front and center.

Finally, I'm happy to jump on a vidchat when I come back from vacation, and discuss some of these topics live. I would also suggest coming to the OpenTelemetry Spec and community meetings if you can make it to them, and voicing your concerns there. I plan on bringing up the data-format-first and other issues you raise here in tomorrows meeting, on your behalf if you cannot attend.

[jmacd]

I think we can close this based on the discussion here, since these two issues appear to be very similar:
#23 (comment)