Submit experiment results

[openflagr]

Submit experiment results

[zhouzhuojie]

It's not a very high priority, and Flagr already collected server-side evaluation logs.

I would imagine this can be useful for the following scenarios:

• Flag evaluation is not at the same time that the experiment begins. For example, loading flags into a mobile app at the bootstrap time, but the actual experiment starts deep down when people actually see the feature.
  • track is the most accurate way of logging that impression.
• Flag evaluation doesn't collect enough context for logging. For example, you mentioned that the time or latency of a feature that can only be collected after its execution.
  • track can help to add that context to the logs.

I will keep this issue open in case someone or me wants to implement it.

[zhouzhuojie]

That sounds like a good idea! And definitely open to contributions!

I think it's similar to track, so clients can send back to Flagr with additional evaluation context together with previous evaluation result.

I would propose

• New API: POST /evaluation/track
• Request body schema can be the same as the response of POST /evaluation
• Additional evaluation context can be added to evalContext.entityContext

https://checkr.github.io/flagr/api_docs/#operation/postEvaluation

Let me know if you want to start working on it, or I will find some time later.

[johnsardine]

@zhouzhuojie Thank you for your prompt reply. I was thinking a bit more with my team and maybe you also reached this conclusion on your side. The actual collection of the results is probably not a priority for this project.

Imagine you collect metrics using Mixpanel, you can easily collect the experiment results there. Otherwise you'd have to develop a couple of analysis mechanisms into Flagr.

Thank you for looking into this but I'll actually drop this request unless you actually see the need for it.

[github-actions]

Stale issue message