gbv/avram-js

Avram Schema Language implementation in JavaScript

avram-js's Issues

Document error format

Validator returns a list of validation errors but format of an error needs to be specified.

Support MARCJSON input format

meta-validator: check restrictions by format family

Check restrictions by format family.

CLI: print recommended installation on help

Document extensions

Add SchemaBuilder

Formerly included as Analyzer

Remove marc-json

This library should be agnostic to MARC or PICA. Preprocessing MARC data to be fed into avram validator is out of the scope of this project

Support command line arguments

• --format
• --values
• --subfields
• --schema
• ...

Release for Avram 0.9.3

Current dev also anticipates some of Avram 0.9.3. Requires https://github.com/gbv/avram/milestone/7 to be released.

Support MARC Example format

Looks like this:

LDR 01000cam a2200265   4500
001     1844949419
245 10 _aKirschblütenküsse auf Mallorca
       _cKeri Anne Arden
935    _iSLoT
951    _aBO
       _bj

Extend SchemaValidator

To include additional requirements of Avram format and possibly give warnings.

Add schema-to-html converter

Update web demo

The web application in docs/ is broken because it required Analyzer (see #4) but this is not the main purpose of avram-js anyway. Better add validation demo instead. See https://csvlint.io/ for a similar application.

Provide error location in defined form

Errors are defined at https://github.com/gbv/avram-js/blob/main/test/suite/README.md and partly documented in main README.md but implementation slightly differs. In addition to consistent definition, include

• marcspec subset tag ( "^" indicator | "$" subfield )? ) ( "/" position )?
• PICA Path subset tag ( "/" occurrences | "/$x" counter )? ( "$" subfield ( "/" position )? )?
• field name

Add Schema class

• to check an Avram Schema (in addition to rules of its JSON Schema)
• to provide methods to get information about the schema
• to convert from and to other formats (e.g. #9 and https://csvw.org/)

CLI: add method to infer/specify record types

Primary use case MARC data. Could be hard-coded in lib/record. In any case, record types are

• either specified for all input records with --type
• or determinde from a plugin function ("recordTypeProvider"), e.g. marc21bibliographicRecordType

Default could be to enable marc21bibliographicRecordType unless --type "" is given.

The provider could be set with -x, --extension [name] where name is a node package or local file. The extension can provide methods detectRecordTypes(fields) as well as rules ( #15), e.g. loadRule returning an object with methods checkRecord, checkField, checkSubfield .

Adjust to 0.9.7

Definition of rule ids has been changed to be IRI references.

Support validation with external rules

Requires a method to load/supply code via plugins and/or a ruleLoader.

• A plugin maps external rules (e.g. xsd:nonNegativeInteger) to executable code
• Plugins can also have parameters (when external rule is given as object). See Shacl-based rules in QA Catalogue for use case
• Add option plugins with name(s) of node modules or relative JavaScript files to be loaded

A plugin (aka rule engine) gets the external rule (string or object) and returns a rule checker. A rule checker is a function that gets a record/field/subfield and returns a (possibly empty) array of errors. An error is a JSON object with keys message. Additional keys error (rule id), tag, occurrence, are set automatically by caller.

Rules at upper level Avram elements are initialized first, so in this case:

{
  "fields": {
    "abc": {
      "rules": [ "x-rule" ]
    }
  },
  "rules": [ 
    { "id": "x-rule", .. }
  ]
}

The rule engine knows about { "id": "x-rule", .. } first so it can use "x-rule" as reference to the former rule.

Test validation of MARC data

Included in t/files and t/schemas but not covered by tests yet.

Support reading PICA files

There seems to be no JavaScript PICA parser so create of port of https://metacpan.org/release/PICA-Data

Add examples in MARC, PICA and CSV

Use of this module can better be illustrated with actual example files. Especially MARC21 should be shown, based in schema files available via https://github.com/timathom/marc-schema.

Common user story: I have MARC21 files, how do I check them against MARC21 format specification?

Add converter to JSON Table Schema

A flat family Avram schema can be converted to a JSON Table Schema. See also https://csvlint.io/about.

Extend class Record with MARC parser

marcjs ignores errors. Maybe https://www.npmjs.com/package/marcrecord works or a proper MARC parser needs to be implemented.

Finish validation

Missing:

• format families
• indicators
• countSubfield
• countCode
• missingField
• externalRule

Then check against the validation rules in the specification

Support loading external codelists via extensions

Add validation option invalidFlag

Fix reading gzped MARC files

This should work but does not:

./bin/avram-analyze test/files/sandburg.xml.gz