zazuko/rdf-vocabularies

Zazuko's Default Ontologies & Prefixes

@zazuko/vocabularies -- Zazuko's Default Ontologies & Prefixes

This package contains a distribution of the most commonly used RDF ontologies (schema/vocab, whatever you call it) including their default prefixes, together with a set of utility functions to work with prefixes.

It is extending RDFa Core Initial Context and contains what we consider commonly used prefixes. Some popular prefixes do not resolve to dereferencable RDF and are thus skipped.

The package is built for use in Node.js projects. We ship N-Quads files of the vocabularies so it could be useful for other programming languages as well as you do not have to take care of downloading the ontologies yourself.

Installation

This is a new release of the previous monolithic package which grew quite a lot.

It is recommended to install the prefixes package and select vocabulary packages individually to reduce the size of node_modules.

$ npm install @zazuko/prefixes @vocabulary/rdf @vocabulary/schema ...

You may still choose to install a one package to get them all.

$ npm install @zazuko/vocabularies

It remains pretty much compatible with the previous @zazuko/rdf-vocabularies package. The main difference is that now it is ESM-only.

Usage

(Read below and take a look at some examples.)

Dataset-as-code modules

All vocabularies published from this package are also exported as JS modules so that then can be imported synchronously (no parsing required) and without additional dependencies when in web app setting (see the raw-loader instructions below).

Modules @rdf-vocabularies/datasets exports factories which returns an array of quads Quad and take RDF/JS DataFactory as parameter.

import $rdf from 'rdf-ext'
import schema from '@vocabulary/schema'

const dataset = $rdf.dataset(schema({ factory: $rdf }))

Vocabularies Metadata

See _index.nq.

vocabularies()

Exported from @zazuko/vocabularies', the vocabularies() function accepts an optional options object:

• options.only: Array?, default: undefined, a subset of all available prefixes, will only load these.
• options.factory: RDF/JS DatasetFactory, default: rdf-ext, a dataset factory abiding by the RDF/JS Dataset Specification, used to create the returned datasets.
• options.stream: Boolean, default: false, whether to return a RDF/JS quad stream instead of regular objects/datasets.

Loading all Vocabularies as Datasets

In browser environment this will cause a request for each individual dataset. It is thus recommended to always only load the needed ontologies to reduce the unnecessary traffic and save bandwidth.

import { vocabularies } from '@zazuko/vocabularies'

vocabularies()
  .then((datasets) => {
    /* `datasets` is:
    {
      "csvw": Dataset,
      "sd": Dataset,
      "ldp": Dataset,
      "schema": Dataset,
      "owl": Dataset,
      "void": Dataset,
      "sioc": Dataset,
      "foaf": Dataset,
      "time": Dataset,
      "dcat": Dataset,
      "oa": Dataset,
      "gr": Dataset,
      "rdf": Dataset,
      "cc": Dataset,
      "ssn": Dataset,
      "rr": Dataset,
      "rdfa": Dataset,
      "org": Dataset,
      "sosa": Dataset,
      "dc11": Dataset,
      "skos": Dataset,
      "dqv": Dataset,
      "prov": Dataset,
      "og": Dataset,
      "qb": Dataset,
      "rdfs": Dataset,
      "dc": Dataset,
      "ma": Dataset,
      "vcard": Dataset,
      "grddl": Dataset,
      "dcterms": Dataset,
      "skosxl": Dataset,
      "wgs": Dataset,
      "dbo": Dataset,
      "dbpedia": Dataset,
      "dbpprop": Dataset,
      "rss": Dataset,
      "cnt": Dataset,
      "vs": Dataset,
      "hydra": Dataset,
      "gn": Dataset,
      "gtfs": Dataset,
      "geo": Dataset,
      "geof": Dataset,
      "geor": Dataset
    }
    */
  })

Loading only some Vocabularies as Datasets

import { vocabularies } from '@zazuko/vocabularies'

vocabularies({ only: ['rdfs', 'owl', 'skos'] })
  .then((datasets) => {
    /* `datasets` is:
    {
      "owl": Dataset,
      "skos": Dataset,
      "rdfs": Dataset
    }
    */
  })

Getting a Readable Stream (Quad Stream)

import { vocabularies } from '@zazuko/vocabularies'
const stream = await vocabularies({ stream: true, only: ['rdfs', 'owl', 'skos'] })

Using vocabularies function in browser

The preferred usage in browser projects is to avoid importing from @zazuko/vocabularies because that will require additional bundling of dynamic n-quads modules.

Instead, import from the partial modules (without even installing the big package, if possible):

• import { expand } from '@zazuko/prefixes/expand'
• import { prefixes } from '@zazuko/prefixes/prefixes'
• import { shrink } from '@zazuko/prefixes/shrink'

The module @zazuko/vocabularies/expandWithCheck requires rdf-ext and parses datasets. See the instructions below for examples how to configure the application.

The package's main module can also be used in browser albeit it needs a bundler such as webpack and additional steps to configure it:

The package can be used in browser albeit it needs a bundler such as webpack and additional steps to configure it:

• Enable dynamic imports. In webpack it is done with @babel/plugin-syntax-dynamic-import

• Extend the bundler setup to have it load the contents of vocabulary files (all n-triples). In In webpack it can be done with raw-loader:

  module: {
    rules: [
      {
        test: /\.nq$/,
        use: ['raw-loader']
      }
    ]
  }
  

• Be careful with prefetching chunks. Some applications may generate prefetch links for dynamically loaded chunks. Some of the ontology files are quite large and their number will grow over time. Hence, it may be desired to exclude certain chunks from the being eagerly loaded. Check the wiki for examples.

Expanding a Prefix

expanding means: 'xsd:dateTime' → 'http://www.w3.org/2001/XMLSchema#dateTime'. It is the opposite of shrinking:
expand(shrink('http://www.w3.org/2001/XMLSchema#dateTime')) === 'http://www.w3.org/2001/XMLSchema#dateTime'

There are two ways of expanding a prefix:

• vocabularies.expand(prefixedTerm: String): String synchronous

  Expand without checks. It is similar to prefix.cc in the sense that prefix.cc would expand schema:ImNotInSchemaDotOrg to http://schema.org/ImNotInSchemaDotOrg.

• vocabularies.expand(prefixedTerm: String, types: Array<String|NamedNode>): Promise<String> asynchronous

  Expand with type checks. types is an array of strings or NamedNodes. See this example:

  const { expand } = require('@zazuko/rdf-vocabularies')
  const Class = expand('rdfs:Class')
  const Property = expand('rdf:Property')
  
  // Will return <schema:person> expanded to `http://schema.org/Person`
  // iff the dataset contains either:
  //   <schema:Person> <rdf:type> <rdfs:Class>
  // or
  //   <schema:Person> <rdf:type> <rdf:Property>
  await expand('schema:Person', [Class, Property])

Shrinking an IRI

shrinking means: 'http://www.w3.org/2001/XMLSchema#dateTime' → 'xsd:dateTime'. It is the opposite of expanding:
shrink(expand('xsd:dateTime')) === 'xsd:dateTime'

• vocabularies.shrink(iri: String): String

  Note: returns empty string when there is no corresponding prefix. Always check the output when using shrink with user-provided strings.

  import assert from 'assert'
  import { shrink } from '@zazuko/prefixes'
  
  assert(shrink('http://www.w3.org/2001/XMLSchema#dateTime') === 'xsd:dateTime')
  assert(shrink('http://example.com#nothing') === '')
  
  const iri = 'http://example.com#nothing'
  const stringToDisplay = shrink(iri) || iri
  console.log(stringToDisplay) // 'http://example.com#nothing'

Accessing Prefixes: vocabularies.prefixes

Getting an object with prefixes and their base URI:
(Returns this object.)

import { prefixes } from '@zazuko/prefixes'

console.log(prefixes)
/*
 {
  v: 'http://rdf.data-vocabulary.org/#',
  csvw: 'http://www.w3.org/ns/csvw#',
  sd: 'http://www.w3.org/ns/sparql-service-description#',
  …
}
*/

Accessing Data Files from the Package

Accessing the N-Quads files:

import path from 'path'
import module from 'module'

const { resolve } = module.createRequire(import.meta.url)

console.log(path.resolve(resolve('@vocabulary/skos/skos.nq')))

Command line

The package also includes a simple command line interface which forwards the vocabulary datasets to standard output. It can be used in two ways.

By prefix:

rdf-vocab prefix foaf

By namespace URI:

rdf-vocab prefix http://schema.org/

Versioning Scheme

The packages follow semver but that is to indicate library code changes rather than vocabulary changes. Usually, that is. If we identify significant changes to the source RDF for any given vocabulary, we may decide to bump a major version.

Adding new prefixes

New prefixes can be added by opening a pull request on Github. For new requests, first check if the creator/owner of the namespace defined a prefix. If not check prefix.cc. In case prefix.cc is ambiguous a discussion should be raised before the pull-requests gets integrated. Last thing to check are the predefined namespaces in the DBpedia SPARQL endpoint or other popular RDF resources like LOV. If you find one please refer to it in the pull request.

Steps to add a prefix

1. Add a new directory under ontologies with a package.json. For a minimal example see ACL vocabulary
2. If necessary, add overrides to the vocabulary key, similar to the others
   • for the file option, a file: scheme IRI can be used, with path relative to the repository root
3. Run npm run fetch in the vocabulary's dir to process the triples.
4. Add a dependency to vocabularies meta package
5. Commit changes and submit a PR

Project-specific prefixes

It is also possible to add prefix within a project so that it can be used with the functions expand and shrink.

import { prefixes, expand, shrink } from '@zazuko/prefixes'

prefixes['foo'] = 'http://example.com/'

// 'http://example.com/bar'
const foobar = expand('foo:bar')

// 'foo:bar'
const prefixed = shrink(foobar)

rdf-vocabularies's Issues

Request for new vocabularies

Namaste all. It is a request for few vocabularies. These are mainly used for implementing solid specifications. These all vocabs have prefix.cc registered prefixes, and ther namespace base iri de-references to their ontology files

prefix	title	description	uri
cert	The Cert Ontology 1.0	Ontology for Certificates	https://www.w3.org/ns/auth/cert#
stat	POSIX stat	Describes terms for POSIX-like files and directory listings	http://www.w3.org/ns/posix/stat#
solid	Solid Terms	The Solid Terms vocabulary defines terms referenced in Solid specifications.	http://www.w3.org/ns/solid/terms#
oidc	Solid OIDC	The OpenID Connect vocabulary used by the Solid-OIDC authentication specification.	http://www.w3.org/ns/solid/oidc#
pim	Workspace Ontology	This ontology is for use in describing Workspaces.	http://www.w3.org/ns/pim/space#

Thanks again for your work

Add prefix & namespace metadata to each file

Good idea via @semanticfire, we should add two more quads to each vocabulary dump to describe the prefix & namespace we use for it.

Bart proposed using vann but I can't seem to fetch an RDF version of it anymore, the links are dead.

Ivan is using the RDFa vocabulary for this, see for example here. So my proposal would be to add these two quads using the RDFa vocabulary.

Extract the code used to generate this package contents (index, etc.)

I just got this idea to extract the code used to generate this package contents (index, etc.) as well as the generator of @tpluscode/rdf-ns-builders

This would allow others to create their own similar packages for other ontologies which we might not want to incorporate in this list for some reason.

Originally posted by @tpluscode in #55 (comment)

High-level Strategy for rdf-vocabularies

Requirements

• Opinionated list of prefixes, and namespaces independently available.
• Opinionated list of snapshotted ntriple dumps, and their definition how it was fetched on their (original) source.
• Optional (added) metadata on the used meta-ontology and its used annotations (rdfs:label vs. schema:name). (RDFS/OWL etc.)

JS Requirements

• Opinionated map of prefixes, and namespaces independently available.
• Opinionated list of snapshotted transformed to RDF/JS quads.
• Defined package structure / define ontologies in a NPM package.

Implementation

• Ontology package -> annotation (prefix, how to get it) -> tiny (<5kb) (as Turtle) [authorative source]

  • updates JS prefix map (transformation)
  • publishes a NPM package with the JS prefix maps

• Snapshots package -> uses the Ontology package

  • fetches (on the web) the snapshotted ntriple dumps (gzipped)
  • publishes a NPM package with the RDF-JS quads versions (only)
    • define an interface that covers the quads and metadata for external people to create compatible ontology packages

Add SemWeb Vocab Status ontology

Dereferencable here: http://www.w3.org/2003/06/sw-vocab-status/ns#

Is it any popular? Hydra uses it to annotate its terms.

additionally, I'm thinking that the prefix server could actually take advantage of this and/or similar annotations to highlight deprecated terms

Issues in namespace list

Issues in current namespace list:

• duv: NS should be http://www.w3.org/ns/duv# according to ontology but different in RDFa initial context, I guess it simply points to the HTML spec
• dc & dcterms point to the same URI. We should settle on one
• dbpedia points to resource and ontology, should be resource IMO

Overlapping prefixes

Overlapping prefixes are not handled as expected (original issue zazuko/SPEX#20).

The following test doesn't pass:

it('handles overlapping prefixes', () => {
  prefixes['foo'] = 'http://example.com/foo/'
  prefixes['bar'] = 'http://example.com/foo/bar/'

  expect(shrink('http://example.com/foo/test')).toBe('foo:test')
  expect(shrink('http://example.com/foo/bar/test')).toBe('bar:test')
})

Note that it works if the prefixes are added in reverse order:

prefixes['bar'] = 'http://example.com/foo/bar/'
prefixes['foo'] = 'http://example.com/foo/'

SIOC

It looks like SIOC is skipped but I find it in ontologies?

submit QUDT to LOV

QUDT is missing from LOV: https://lov.linkeddata.es/dataset/lov/terms?q=qudt. If as part of #13 you dig through the structure of QUDT, it would be great if you submit it properly to LOV.

New ontology: EBUCore

There is a link to the RDF/OWL here https://tech.ebu.ch/MetadataEbuCore

Using the package in browser environment

I would like to be able to use this package's functionality in the browser. Currently the biggest obstacle is using node.js filesystem APIs (fs and path).

I propose to split the ontology code from the loading code so that it can be substituted with different code for the browser. I have two ideas:

1. Simpler, to use fetch. The problem here is that the files have to be fetched from somewhere. Either raw.github URLs or manually added to the deployed app.
2. Use dynamic imports, which forces some additional effort on the consumer to set up webpack or similar to handle

Add override for XSD namespace

The one big namespace missing is XSD, which seems surprisingly hard to get. I asked around on Twitter here and after an interesting thread @gkellogg pointed to some work he did here.

This should be added to overrides.js

Add SHACL

Somehow we completely missed SHACL

https://www.w3.org/TR/shacl/

New Ontology request - GS1

Hi there.

hey I'm building a very comprehensive seo plugin for wordpress. And I absolutely love this git. So with that said. Ive' been researching new ontologies and specifically GS1 since they've been recognized by schema and W3. But they are not registered on http://dbpedia.org/sparql?nsdecl. But heres what I got for your reference.

https://www.gs1.org/standards/gs1-source/guideline/gs1-source-tsd-technical-implementation-guide-aggregators

https://www.gs1.org/edi-xml/technical-user-guide/Schema_hierarchy

https://www.gs1.se/en/our-standards/Technical-documentation/code-lists/
https://www.gs1.se/esap/20tradeitem/ver285/gu1.htm
https://www.gs1.se/esap/20tradeitem/ver285/gu5.htm
https://github.com/mobilemadman2/gs1

New ontology: dcmitype

https://dublincore.org/specifications/dublin-core/dcmi-terms/

Bug: buildPath returning incorrect path

buildPath is returning node_modules/@zazuko/ontologies instead of the correct node_modules/@zazuko/rdf-vocabularies/ontologies.

Additionally, the vocabularies() function should properly throw an error if it reads files in the wrong location.

Thanks for this tool!

Update request: New release for m4i

Dear zazuko team,

there is a new release for Metadata4Ing: https://w3id.org/nfdi4ing/metadata4ing/1.2.0/

We wanted to let you know and ask whether you could update here/ for the zazuko prefix service.

Thank you and all the best form the m4i working group

Susanne

Add geonames

Used in Swisstopo dataset:

@prefix gn: <http://www.geonames.org/ontology#> .

RDF Ontology: http://www.geonames.org/ontology/ontology_v3.1.rdf

Add rdfa

http://www.w3.org/ns/rdfa#

And while we are at it, vann: http://vocab.org/vann/

New ontology: metadata4ing

Hi zazuko team - is it possible to include the metadata4ing ontology in your prefix lookup service? You can find details on the ontology here:

http://metadata4ing.org

Serializations in different formats are also available via that link.

Thanks and kind regards, Marc

Evaluate integrating QUDT

Evaluate what it means to integrate the various QUDT schemas and ontologies.

• http://www.linkedmodel.org/catalog/qudt/1.1/index.html
• http://www.qudt.org/release2/qudt-catalog.html

Questions:

• are there clashes between the two versions?
• why are they split among files even though there seems to be only one namespace used?
• what namespaces & files should be sourced?

Using expand requires raw-loader

When the @zazuko/rdf-vocabularies/expand module is imported in a web project, webpack will try to create chunks for all the vocabularies because the expandWithCheck is dynamically loaded and it pull the n-quad sources.

With raw-loader added the bundle sizes explode.

Probably the only way would be to separate the expand and expandWithCheck modules. What do you think @vhf?

Add Simple Event Model

Spec: https://semanticweb.cs.vu.nl/2009/11/sem/
Prefix: sem
Namespace: http://semanticweb.cs.vu.nl/2009/11/sem/

New ontology: EXIF

See https://www.w3.org/2003/12/exif/

Schema (RDF/XML) is here: http://www.w3.org/2003/12/exif/ns

Add location core vocab

I just saw this on Bart's demo: https://www.w3.org/ns/locn

I see that we don't have this one

Extract prefix list into its own library

@zazuko/rdf-vocabularies is quite large (nearly 20MB). tpluscode/rdf-ns-builders@3019d71 was committed to avoid forcing this as a runtime dependency, but there are still cases where it would be useful to have a simple library containing just prefixes.js to avoid having to hard-code the IRIs elsewhere (eg libero/article-store#260 (comment)).

Would it be possible to extract this into its own package? (@zazuko/rdf-vocabulary-prefixes or something?)

Make ontologies available in standalone repository, to include with git sub modules

Thanks for great work.

It would be great if ontologies folder itself is available as separate git repository. We can still include it as a folder for all practical purposes using git-sub-modules. This will enable to include same stack-agnostic ontologies in any repository with any language stack.

Add Bibliographic Ontology (BIBO)

https://www.dublincore.org/specifications/bibo/bibo/

Add SDMX namespace

http://purl.org/linked-data/sdmx#

DBpedia prefixes

I thought we might add prefixes for the non-vocabulary DBpedia namespaces:

{
  "dbr": "http://dbpedia.org/resource/",
  "yago": "http://dbpedia.org/class/yago/",
  "yago-res": "http://yago-knowledge.org/resource/"
}

Not too sure about the YAGO stuff though...

Adjust GS1 source

See https://twitter.com/philarcher1/status/1446501549296001025

Add http

Spec:

https://www.w3.org/TR/HTTP-in-RDF10/

content-neg seems to work

W3C Block

Apparently we get blocked fast by w3c.org while fetching the data. @vhf how fast/when does it happen? Maybe @iherman can help us out here.

Ivan, we fetch RDF vocabularies in an automated way every few months and add the latest RDF versions of it in this repository. That seems to be enough to get the IP blocked by W3C for a while.

Wrong namespace for EARL

The vocabulary should use http URLs but we have https

Add ical

Via @iherman I got links to the following RDF resources, taken from his mail:

• as: https://www.w3.org/ns/activitystreams-owl.ttl
• ical: http://www.w3.org/2002/12/cal/icaltzd.rdf (it downloads the RDF/XML file in most browsers instead of displaying it)
• xhv: http://tinyurl.com/yy4pbocc (the vocabulary is in the HTML file in RDFa, and this URL runs the RDFa distiller to return the turtle file)

Those should be sourced as well

Add RDA elements namespacer

More info & namespace: http://www.rdaregistry.info/Elements/u/#

MIA prefixes

Those prefixes & namespaces are defined in RDFa initial context but are missing in action, or at least an RDF version of them is. Please comment if you find any of them.

• ctag:
• gml:
• rev:
• rif:
• sf:
• v:
• xml:

ebucore prefix is wrong

This:

rdf-vocabularies/src/prefixes.ts

Line 21 in d42929d

ebucore: 'https://www.ebu.ch/metadata/ontologies/ebucore/ebucore.rdf',

Should be

http://www.ebu.ch/metadata/ontologies/ebucore/ebucore#

Add XKOS

Turtle: https://github.com/linked-statistics/xkos/blob/master/xkos.ttl

Rename dc11 to dc?

Do we want to rename dc11 to dc?

Add FRBR Ontology

More information: http://www.sparontologies.net/ontologies/frbr

schema.org download location

According to schemaorg/schemaorg#2304 (comment) the download location here:

rdf-vocabularies/overrides.ts

Line 108 in d026ae4

schema: { file: 'http://schema.org/version/latest/schema.nt' },

Might change. We need to keep an eye on that issue.