JSLT does not produce itself: empty objects

[schibsted]

JSLT does not produce itself: empty objects

[Mohammadcore]

@larsga I haven't got the time to test it yet. But from the quick look I had at the code changeset, looked fulfilling the requirement. Thank you very much.

[larsga]

Yes, this is expected. When you produce values with queries it's quite common to end up with null, empty arrays, empty objects, and so on. In these case JSLT simply leaves out the keys that have empty values rather than produce JSON objects that are full of junk.

[vectro]

This rationale makes sense to me, although i can think of use cases where I would want to produce a document like that. For example, I may want to produce a document like {"fooList": []} so that I don't have to special-case in the reader the situation where there are no items.

At the least this behavior should be prominently documented, IMHO.

[larsga]

Yeah, I agree that documentation is the solution for now.

I was thinking of adding an option to let people specify a filter to say what keys they want included, so that if you want to keep all keys no matter what the value is you can include a filter that says true. For example.

[vectro]

I think your example got cut off?

How about a function (or keyword) preserve() which is just like the identity function except that its result is never cleaned up? I can imagine how this could be hard to implement though, as it would require tagging the json objects somehow during construction.

[larsga]

The example is before the "For example." :)

Not sure how the preserve() would work. You mean something like:

{
  "foo" : preserve(.doesntexist),
  "bar" : .doesntexisteither
}

resulting in

{
  "foo" : null
}

?

Does that really make sense? Why is it important to keep the null in one case, but not in the other? Does it really make sense to require the null at all? So far my preference has been to just put this off until I get a better understanding of what kinds of usecases people actually have.

[vectro]

I'm not sure that I understand your proposal. Perhaps you could provide a full example? You have correctly understood mine. I'm not going to defend why somebody might want foo but not bar; I just think it's desirable to be able to produce any wanted output object.

As far as the specific use cases that might require this, the one that's in front of me is the desire to keep the types of things the same even for edge cases. So that I can guarantee a particular JSON path will always lead to an element of a certain type (be it an object, arary, etc.). This simplifies the code path on the reader side because you can just iterate whatever container you expect to be there without having to special-case null etc. My use case is not that compelling because I have control over the reader codebase so I can add that complexity there. But what if I didn't?

[larsga]

When we build objects from object expressions we filter the key/value pairs to decide which ones to leave in, and which ones to leave out. Here's the line that does the filtering: https://github.com/schibsted/jslt/blob/master/src/main/java/com/schibsted/spt/data/jslt/impl/ObjectExpression.java#L60

That NodeUtils.isValue() method is a predicate that matches what I think makes sense to include. But other people might have other ideas.

So I thought: why not let them plug in their own filter? Basically a JSLT expression that, when it evaluates to true keeps the key/value, and when it's false drops it.

The current filter can be replicated by:

. != null and not(is-object(.) and size(.) =0) and not(is-array(.) and size(.) =0)

With my proposal someone could say

. != null

That is, leave out only nulls, and keep everything else. Or they could add in ... and != "" and ditch empty strings, too.

I just think it's desirable to be able to produce any wanted output object.

I think I mostly agree with that, but the question becomes what is really wanted? And I'm not sure I know the answer yet.

As far as the specific use cases that might require this, the one that's in front of me is the desire to keep the types of things the same even for edge cases. So that I can guarantee a particular JSON path will always lead to an element of a certain type (be it an object, arary, etc.).

As in, you want to ensure that there's always {"foo" : [ ... ]} regardless of whether the ... is nothing, or some sequences of values? To my mind {"foo" : []} and {} are semantically the same thing. But I agree people might have use-cases where that's not quite true.

[vectro]

That NodeUtils.isValue() method is a predicate that matches what I think makes sense to include. But other people might have other ideas.

Makes sense to me. Only downside that I see to this approach is that the predicate may not be invoked with all the necessary context, if the rule is complex.

To my mind {"foo" : []} and {} are semantically the same thing.

Of course we had a koan about this, but I agree, they're semantically the same. Still, they are lexically different, a distinction which might arise sometimes.

[larsga]

Only downside that I see to this approach is that the predicate may not be invoked with all the necessary context, if the rule is complex.

I guess it's a bit of an open question if a rule that complex makes sense.

Your point about the code safety convenience (from the koan) is probably something that people do sometimes, but whether they should is less certain.

But, yes, where there is a distinction people might care about it.

For now I'm still inclined to wait for a concrete use-case.

[Mohammadcore]

{
  "foo" : preserve(.doesntexist),
  "bar" : .doesntexisteither
}

resulting in

{
  "foo" : null
}

This would be something we highly are in need of for a project we are working on.
is there any plans to add this feature to the library?

[larsga]

@Mohammadcore See the response to the ticket you just opened: #51

[larsga]

I consider this solved, now that we have configurable filters for which values to leave out. Please reopen if you disagree.