externalize context from JSON instances

[webb]

externalize context from JSON instances

[TJKoury]

What would this accomplish? Context is critical to validating data, segregating it into a separate document would simply increase the workload to validate the message. Is there a recommended workflow for consuming LD payloads that includes preloading context data?

[webb]

Yes. See https://json-ld.org/spec/latest/json-ld-api-best-practices/#cache-context ("Best Practice 3: Cache JSON-LD Contexts"), which recommends reusing contexts.

Also https://json-ld.org/spec/latest/json-ld/#interpreting-json-as-json-ld ("4.9 Interpreting JSON as JSON-LD").

Basically the idea is that JSON-LD should be able to look like plain uninteresting JSON, and the LD needs to be accessible somehow. You can put the context inline, link to a context via a URI, or include the context in an HTTP header.

We need to avoid the appearance that people need to jam contexts into their instances; we want the JSON to seem as simple as possible.

[webb]

Yeah, I see you. I've left this open because I'm focusing elsewhere right now and will be digging into the JSON-LD depths in July.

I don't know that we need to explicitly put in @ids for QNames (e.g., nc:Person), since the URI for a QName would be the expansion of the prefix (nc -> http://release.niem.gov/niem/niem-core/3.0/#) concatenated to the local part (Person) yeilding the URI (http://release.niem.gov/niem/niem-core/3.0/#Person), which is what we want.

We could put @type on properties in a context, but I'm not sure what that would buy us at this point. I'm going to be digging into JSON-LD processors, but outside the RDF world (like Jena) I'm not sure how @type gets utilized.

First cut on a default context for NIEM 4 will just be declaring namespaces. Later, I will dig into Jena-stack processing of NIEM data, using RDFS declarations corresponding to NIEM schema definitions, per the NDR section 5. Any help is welcome, especially if you want to get ahead of me on that.

[TJKoury]

I opened an new issue just to start a conversation, but since you're here....

Here's what @type buys me specifically:

I'm writing a framework that will walk the inheritance tree for any message received all the way back to basic types, then create persistence schemas automatically.

So if a message is received, and as you said above the context is available either from:

• The message itself
• Cached reference
• Externally resolvable reference

My framework will walk the types back to the basic types, then create an persistence mapping from that, whether that's an SQL statement, or ElasticSearch index, or file format, or whatever.

For example:

nc:PersonType -> nc:PersonNameType -> nc:PersonGivenName -> nc:PersonNameTextType -> nc:ProperNameTextType -> nc:TextType => (sql engine) varchar(255)

The framework will be able to generate a persistence mapping to add to a @vocab doc.

Right now I'm using a custom version of sqlite3 for testing with my framework, which is called Union-Station (for reference).

[TJKoury]

So it's a lot like Jena (what you reference above), except written in NodeJS and C++, with a lot less code and a visual graph interface like Node-Red .

The current demo version is actually a heavily modified fork of Node-Red, but the next version is a ground-up rebuild.

[webb]

Everything you're saying sounds great to me. Anything I can do from the NIEM-standardization side to help? I don't want anything we do to get in the way of this kind of thing.

[TJKoury]

At this point just answering my questions is a huge help! I do appreciate the quick feedback.

There isn't really anything that is going to help / hurt from the standardization side, it's just when I'm trying to write code sometimes there are holes in my understanding that I'm trying to fill. For example, this conversation is informing this code starting around line #173.

I don't have all the code in the open yet but it's coming.

[webb]

A draft of the proposed default context for NIEM 4 is at https://github.com/webb/NIEM-Releases/blob/dev-json-ld-context/niem/context/niem-4.0.jsonld. Take a look if you get a chance.