End to End Flow

[PhanLe1010]

End to End Flow

[yasker]

I meant the user needs to know the differences, and they need to know there must be at least one field (IIRC). Those concepts are tie to InfluxDB. We can still switch to another DB later, but users always need to follow the InfluxDB concept for tag vs field, no matter what's the backend DB is.

It's not a major issue. As long as we explain what is a tag and what is a field and how users should choose between the two for things they want to record, that would be fine.

[vincent99]

The top level object you send should be a map. If you need to add some more info later and it's an array you're in for a bad time. {"metrics": [ ... ]}

...for example, tags/install uuid seems like it could be sent once at the root instead of duplicating with every data point. So you could add a top-level "tags" map that applies to every metric. But if your top-level is an array, you're just stuck because there's nowhere to put it and will have to break the API.

Think about data types, are fields values always an integers or can they be sent as strings or bools, etc? I don't know what I'm allowed to send. Handling very large numbers (> 2^52) in JSON is error-prone/implementation-specific. If "storage capacity" is in bytes that's "only" 4,500 TB (half a rack of backblaze storage pods) before some clients can't accurately represent it. But losing a bit of accuracy for a metric is probably not a critical problem unless you try to compare/find exact data points.

When should I (as a person adding this to my product) send multiple top-level entries in the array versus one entry with multiple "fields"? Assume I (a person reading the docs) know and care nothing about Influx and how it works.

Same for "tags", when do I want a tag vs a field? Calling them "labels" might make more sense to our k8s-centric crowd.

For rancher we often look at the min/mean/median/max value for a metric, to get an idea of the distribution for how a feature is used. I assume you can calculate that out of influx somehow.

[yasker]

What are the differences between tag and field? By defining those in Metric we're tying ourselves to Influxdb (which is also a valid option).

[PhanLe1010]

@yasker

tag are indexed in the InfluxDB for faster lookup and grouping while fields are not. tags are usually used for things that have many different values (e.g, the UUID) so that searching doesn't require scanning over the whole database. Application developers are the ones who know which information are tags and which information are fields.

defining tag and field doesn't tie us to InfluxDB since we can implement the PostgresSQL database client to convert all tags and fields into regular columns when we add records data to the database. Application developers can then manually create indexes on the tags column inside PostgresSQL

[PhanLe1010]

@yasker

I see. Thank you! I will add a note to the README.md when I create it.

[PhanLe1010]

Thanks @vincent99!

• Yes, I will make the top level object a map. Great point!
• Data types can be: string, bool, float 64-bit. (It is possible to support int64 but I am not sure if it is needed). Yeah, it is up to the app developer to choose the unit for fields. As you mentioned, if "storage capacity" can be more than 4500TB, app developer might want to consider using MB as unit instead of byte.
• App developer need to know how InfluxQL works since they will design the measurement (the table) schema in InfluxDB. They will also be the one who design continuous queries to extract useful information out of the raw measurement. They will also write InfluxQL query to create Grafana dashboard. So, knowing InfluxQL is required. But sure, I will have some document and example about how to choose between tags/fields; multiple metrics or a single metric with multiple fields.
• Yes, we can calculate basic stats min/max/mean/median using InfluxQL https://docs.influxdata.com/influxdb/v1.8/query_language/functions/

[tybalex]

does user's application create its own random instance_uuid and maintain it?

[PhanLe1010]

@tybalex Yeah

In Longhorn, we are thinking of using namespace UUID where Longhorn is installed to be the UUID of the Longhorn instance.