add version key to json schema

[perftool-incubator]

add version key to json schema

[rafaelfolco]

Clarifications:

• Multiplex schema should also be versioned or just rickshaw (bench-params.json) ?
• Multiple schema version should match rickshaw's one ? multiplex schema is for mv-params, rickshaw's is for bench-param.json.
• version == "$id" ?
• version could be an integer: .../schema/bench-params-1.json ?
• Older versions of schema should be kept in the schema dir

If I understand this feature correctly, multiplex should add the $id key to bench-params.json, which points to the schema version that validates it in rickshaw project,$id in rickshaw's schema, https://github.com/perftool-incubator/rickshaw/blob/master/schema/bench-params.json#L3:

"$id": "https://raw.githubusercontent.com/perftool-incubator/rickshaw/master/schema/bench-params.json",

So everytime bench-params.json structure is changed in multiplex, $id needs to change in rickshaw.

The resulting rickshaw's bench-params.json should be versioned like:

"$id": "https://raw.githubusercontent.com/perftool-incubator/rickshaw/master/schema/bench-params-1.json",

The resulting multiplex's schema.json should be versioned like:

"$id": "https://raw.githubusercontent.com/perftool-incubator/multiplex/master/schema/bench-params-1.json",

Note that today schema.json is in root dir and it's not versioned.

@k-rister please clarify.

[k-rister]

I think the workshop example is actually better (after looking at the rickshaw ones I think we should consider updating them with arrays of enumerated strings rather than regex patterns -- its just simpler and easier to understand). Here is the relevant section of the workshop schema dealing with the version:

https://github.com/perftool-incubator/workshop/blob/master/schema.json#L67

And here is an example of userenv that is compatible with that schema:

https://github.com/perftool-incubator/rickshaw/blob/master/userenvs/centos8.json#L3

We decided to use dates as our schema versions -- the idea being that is easy to track their order and when there were created. In the case of workshop, the allowable versions is controlled by an array of enumerated values. The way the versions are currently defined is this: the oldest version allowed is the one that defines the basic absolute requirements; subsequent allowed versions add some type of optional allowable field, property, value, etc. If the optional element(s) is/are not present then the "original" (ie. oldest) allowed schema version still defines the minimum requirements that a JSON must meet to pass schema validation.

If we decided to add something to the schema that was a new required element that would necessitate a reset of the schema versions -- all previous version strings would be removed from the enumerated list and a new version string (for "today", whatever day that is) would be created. From that point on any JSON would be required to contain that version to pass schema validation -- so anything with an older version string will automatically fail (note that the JSON would likely fail the version string validation and all fail due to lacking whatever the newly required element is -- the version string just makes it a bit easier to track).

However, if we decided to add something to the schema that is optional the schema would just needed to be updated to reflect that optional change and a new allowable version would need to be added to the enumerated list (again with "todays" date being the new version string).

[atheurer]

I believe what @k-rister described is correct, and we just have not been tracking this in rickshaw as well as workshop. FWIW, I am fine with doing aa full reset for the rickshaw stuff, starting with only 1 new version that is supported.