Giter Site home page Giter Site logo

gs1combinators's Introduction

GS1 Combinator Library

A library to parse the GS1 Events into Haskell data types.

Documentation

View the Haskell API documentation here

Alternatively, the documentation can be generated locally using the following command

./generate_docs.sh

and then viewing docs/index.html in a browser.

Getting Started

Compiling

Download and install the Haskell build tool, stack, and run

stack build

Running Tests

You must always remember to build before testing, since the automatic building before the test building has been removed.

This is automated in run_tests.sh

./run_tests.sh

To perform a clean beforehand, use the -c option

./run_tests.sh -c

To clean, run stack clean

To build, run stack build

To test, run stack test

To parse an XML file, run stack exec Parser-exe -- /path/to/XML/file

More than one file can be parsed at a time. Just append to the arguments.

To check the validity/expected output of the parsing of an XML file in terms of GS1 vocabulary, use this tool. You may be asked to create an account.

GS1 Standards

The following standards and documents have been consulted while developing this library:

gs1combinators's People

Contributors

a-stacey avatar bigdaddy56789 avatar mperry-uni avatar paulsamways avatar sajidanower23 avatar sarafalamaki avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

Forkers

sarafalamaki

gs1combinators's Issues

make a better validation on the GLN

  • removes show from tests, use proper eq
  • adds IllegalFormat error and adds function specifically verifies format (previously it will throw exception)

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Tests isolation sa

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Add more tests

Add more tests to Parser.hs

Original Author: zz-per276

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

xml files from Neil

Original Author: fal05c

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Cursor API loads the entire XML document into RAM

For the purposes of parsing XML, the Cursor API of the package xml-conduit is being used.

Cursor parses the entire document in one go and loads it all into memory, which means that large documents (we are talking Gigabytes) might cause a problem. In the case of an Operating System error in production environment, we may have to move toward using a streaming API (which loads and parses the document line by line).

It is worth refactoring most of the code in the Parser module so that making that shift is easier.

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Tests fixup mattt

Original Author: zz-per276

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Parallel Parsing

Parsing different events in an XML file do not depend on each other.
In particular, parseEventByType is a function that can do with parallel execution.
In a new pass, it is worth looking into making this optimisation.

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Spec doesn't test all tests

If a test fails with HSpec, in Spec.hs, the subsequent tests do not run. This is not optimal - libraries such as Python3 unittest will run all tests even if prior ones fail. This should be fixed.

Original Author: zz-per276

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Quantity for LGTIN Labels

At the moment, the XML standard puts the quantity of an LGTIN label as a different tag. It might be worth encoding the quantity in the URI.

Ideally, the URI for LGTIN labels should look like:
urn:epc:class:lgtin:CompanyPrefix.ItemRefAndIndicator.Lot.Quantity
At the moment, it looks like
urn:epc:class:lgtin:CompanyPrefix.ItemRefAndIndicator.Lot

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Implement SGTIN EPCs

We need this type of EPC for instance level identifiers.

While it is possible to include class AND object level identifiers (instance: SGTIN, lot: LGTIN: class: GTIN) into an EPCIS EVENT, there should only be one identifier โ€“ at the finest available granularity โ€“ representing any given object or group objects in the WHAT dimension; for example, if a trade item is serialised, it should be identified in the WHAT dimension by SGTIN.

Original Author: fal05c

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Refactor design to work with stream

As discussed, refactor the design to work with a stream of data rather than just an entire file. This is because the document reading function reads in entire file in Haskell and so may become inefficient.

Original Author: zz-per276

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Text to String

The codebase converts Text to String a fair number of times, which is not a cheap operation, not to mention - we should not be using strings to begin with.

Unless there is a good reason to, all String occurrences should be Text.

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Parse Event

  • Code refactory: merge smaller modules into their corresponding dimensions, making it easier to prevent import issues -- types will accessed through dimension.

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

dispositionValidList

On page 24 of 62 page CBV doc, the third column of the Disposition table is example, it might not be adequate to serve the purpose of validating a (disposition, business step) pair.

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Quantity is not being parsed properly

It is possible that URIs that have a Quantity element to it are not being parsed properly.

The function parseQuantity looks into the very first occurrence of "quantity" and "uom" and returns that.

One possible fix is to go down a particular node in a Cursor and then return a list of Maybe Quantity for that node.

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Commented out all of Event.hs

Moved Event related stuff to Event.hs
Added Event.hs to GS1Combinators.cabal
Changed import directive in Main.hs because there's no longer anything left in GS1.hs

Original Author: fal05c

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

parser

  • use cursors as the basic tool for parsing
  • the xml must follow the standard, no user vocabulary is allowed
  • added test xml files

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

finish DWhat tests

Original Author: zz-per276

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Split Event to multiple files

The Event file is large, and its size is growing.
Split it into smaller modules, and make them loosely coupled.

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Continuous Integration

Since contributors are actively pushing to the repository, it is important to get CI integrated in the project so that we do not end up pushing something to production that breaks the build.

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Standard and User Vocabulary

Distinguish between Standard (reserved) and User-Defined Vocabulary.

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Make vocabularies extensible

We'll need additional vocabulary elements for each deployment. How do we plug them in? Currently we have things like BizStep and Disposition as types, so adding to them isn't possible. Should they be type classes instead?

Original Author: fal05c

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Tests fixup mattt

Original Author: zz-per276

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Implementation of some EPCs

Contains untested implementation of the following EPCs:
ClassLabel
InstanceLabel
Location
SourceDestType
BusinessTransaction

Further contains a minor bugfix in readURILocationEPC:
The larger pattern was being matched first, making the smaller pattern
redundant and hence generating a warning from GHC. It has been taken care of in this Pull Request.

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Parser work matt

Original Author: zz-per276

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Parser work matt

Original Author: zz-per276

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

URI/private URN/Http URL

All the three mentioned forms of representation should be implemented as typeclass. URI has been done, and the rest of the 2 should follow the similar style.

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Create constructor functions

  • create constructor functions for some types start with mk
  • parse time
  • generalise some functions in Utils.hs

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

New tests

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Bugs to fix after seeing tests fail (3 bugs)

"parse XML to obtain DWhere finds all the dwhere" is failing because the SGLN extension field uses a 0 to represent no extension, but the parser considers the extension to have value of 0

"parse DWhat parses a valid ObjectDWhat" only seems to grab first instance in list of IL, the other 1 expected is in file

"parse DWhat parses a valid AggregationDWhat" also appears to be failing due to a bug in the src

Original Author: zz-per276

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Implement readURI of LabelEPC

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

remove epcistime and exception

  • Use MonadError to deal with the error handling
  • Use built-in time whenever possible

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Consider remove dimension as type

I would consider removing DWhat, DWhen, DWhy and DWhere as component of Event and make them as functions.

dWhat :: Event -> DWhat
dWhat (ObjectEvent a b) = ObjectDWhat a
dWhat (OtherEvent a b c) = OtherDWhat a c 

If these types are kept in the Event, they would create hidden obstacles for parsing. We might need them later, but I think it would be better to be as close to the xsd as possible.

Please let me know what you think about it. ; )

@fal05c

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

XML & XSD files for parser implementation and testing

I've copied these out from the documentation provided. The XSD file probably has a few errors from copying from a pdf doc. It's a starting point though.

Original Author: fal05c

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Fixed Tests

The existing tests have been tentatively fixed in this branch.
More tests need to be added eventually.
The tests for Parser are not fixed.

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Refine by xsd

It refines the data constructors by the gs1 xsd.
It facilitate the process of object creation and enforce the check like EPC/GLN.

If we agrees upon the changes, the rest of the work will be based on this one.

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Use `Maybe` in `Why`

  • I am not sure if we need to use Maybe in Why.

PS

  • I changed the error to MonadError in the why function as well.

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Parser work matt

Original Author: zz-per276

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

add some more implementations

  • add Object
  • add URI
  • add EPC

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Some Label EPCs are not implemented

Label EPC has been refactored into ClassLabelEPC and InstanceLabelEPC.
The respective readURI function for some valid urn's are not considered.
One such example is
urn:epc:idpat:sgtin:4012345.066666.*

This should be a ClassLabelEPC but the Parser fails to recognise this.

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

CBV-Compatible and CBV-Complaint documents

The section 5 of CBV documentation has identified the standard of the CBV-Complaince and CBV-Comaptibility documents.
I think we can to implement it, after we have the document/event ready. as it contains lots of constraints about the input/output of the document, and the URI.

I will keep this issue opened, and summarise the conditions in the comments as follows.

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.