Sort keys of objects

[yamafaktory]

Sort keys of objects

[yamafaktory]

Hi @jameschenjav and thanks for the positive feedback!

Basically jql keeps the insertion order because the preserve_order feature flag was introduced to avoid the default behavior of serde_json (#23, https://github.com/yamafaktory/jql/blob/main/Cargo.toml#L41).

Nobody complained about it so far... and I was the one introducing this (no issue).

So basically, we have two options:

1. remove the flag and then everything will be sorted auto-magically
2. introduce a sort flag

[jameschenjav]

Hi @yamafaktory

Thanks for your response.

It makes sense to keep original order by default. As it will be a breaking change, if someone had already built test-cases without proper comparison, the tests will break.

So imo better to have a new flag.

[yamafaktory]

A follow-up on this topic.

The current issue is that, as already mentioned, serde_json uses a flag to either internally process the data as a BTreeMap or an IndexMap (https://github.com/serde-rs/json/blob/master/src/map.rs#L30-L33). I haven't seen so far a simple way to bypass the flag limitation - basically being to switch from one to another data-structure. Also, to implement a custom sorting from scratch sounds like a poor idea as data can be deeply nested and performance will be pretty bad.

Still trying to investigate a bit. Let's keep this open for now.

[kmaasrud]

I've been investigating this a bit myself, and as you say, there is no way to conditionally toggle a crate feature, since features get applied at build-time.

However, I think Cargo can handle compiling two different versions of serde_json - one with the preserve_order feature and one without (and then simply naming them differently.) I can try this out in a PR if you want, and then we can test if there are any performance or bloat implications.

I personally think the best way is to put optional sorting behind a feature flag in jql. Feels less bloated. Though perhaps we can even hide the sort-flag double-dependency shenanigans behind a feature-flag as well?

[yamafaktory]

@kmaasrud I thought about that too but that would mean two different versions since this is a binary (this can works fine for a the library part of course). Folks that will install it will still get the default one.

[kmaasrud]

@yamafaktory, yes it seems like my idea is not possible.

The thought was to have two versions of serde_json compiled into the binary - one with the preserve_order feature and one without. That would allow us to do e.g. serde_json::from_str::<Value>(json_content) and serde_json_ordered::from_str::<Value>(json_content) based on a condition, where serde_json and serde_json_ordered are the same crate with different features. However, Cargo only allows multiple versions of the same crate if they are indeed DIFFERENT VERSIONS (e.g. 1.0.79 and 1.0.78.) That would be less than ideal - to put it mildly.

I say document the sorting behavior and then people can install their preferred version of jql with cargo install jql --features sorting, for example.

[yamafaktory]

That's an option indeed. My main concern is that it feels a bit awkward and might be even more weird regarding the packages distributed on different OS (not maintained here).

[yamafaktory]

Let's maybe keep this open for a bit longer and see if some trick might help fixing that.

[kmaasrud]

Let's maybe keep this open for a bit longer and see if some trick might help fixing that.

Sounds like a good idea 👍