data61 / gs1combinators Goto Github PK

View Code? Open in Web Editor NEW

5.0 5.0 1.0 844 KB

A library to parse the GS1 Events into Haskell data types

License: Apache License 2.0

Haskell 99.36% Shell 0.64%

gs1combinators's People

Stargazers

Watchers

Forkers

sarafalamaki

gs1combinators's Issues

URI/private URN/Http URL

All the three mentioned forms of representation should be implemented as typeclass. URI has been done, and the rest of the 2 should follow the similar style.

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

New tests

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Parsing different events in an XML file do not depend on each other.
In particular, parseEventByType is a function that can do with parallel execution.
In a new pass, it is worth looking into making this optimisation.

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Some Label EPCs are not implemented

Label EPC has been refactored into ClassLabelEPC and InstanceLabelEPC.
The respective readURI function for some valid urn's are not considered.
One such example is
urn:epc:idpat:sgtin:4012345.066666.*

This should be a ClassLabelEPC but the Parser fails to recognise this.

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

implementation

Issue https://github.csiro.au/Blockchain/GS1Combinators/issues/7

add types specified in docs

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Quantity for LGTIN Labels

At the moment, the XML standard puts the quantity of an LGTIN label as a different tag. It might be worth encoding the quantity in the URI.

Ideally, the URI for LGTIN labels should look like:
urn:epc:class:lgtin:CompanyPrefix.ItemRefAndIndicator.Lot.Quantity
At the moment, it looks like
urn:epc:class:lgtin:CompanyPrefix.ItemRefAndIndicator.Lot

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

make a better validation on the GLN

removes show from tests, use proper eq
adds IllegalFormat error and adds function specifically verifies format (previously it will throw exception)

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Spec doesn't test all tests

If a test fails with HSpec, in Spec.hs, the subsequent tests do not run. This is not optimal - libraries such as Python3 unittest will run all tests even if prior ones fail. This should be fixed.

Original Author: zz-per276

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Split Event to multiple files

The Event file is large, and its size is growing.
Split it into smaller modules, and make them loosely coupled.

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

finish DWhat tests

Original Author: zz-per276

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Parse Event

Code refactory: merge smaller modules into their corresponding dimensions, making it easier to prevent import issues -- types will accessed through dimension.

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Add more tests

Add more tests to Parser.hs

Original Author: zz-per276

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Refactor design to work with stream

As discussed, refactor the design to work with a stream of data rather than just an entire file. This is because the document reading function reads in entire file in Haskell and so may become inefficient.

Original Author: zz-per276

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Address design issues Tony raised

https://github.csiro.au/Blockchain/GS1Combinators/commit/1fa64484263246508953af764728194513ddce1b

Original Author: fal05c

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

split into multiple modules and use lens for some data types

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Refine by xsd

It refines the data constructors by the gs1 xsd.
It facilitate the process of object creation and enforce the check like EPC/GLN.

If we agrees upon the changes, the rest of the work will be based on this one.

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Consider remove dimension as type

I would consider removing DWhat, DWhen, DWhy and DWhere as component of Event and make them as functions.

dWhat :: Event -> DWhat
dWhat (ObjectEvent a b) = ObjectDWhat a
dWhat (OtherEvent a b c) = OtherDWhat a c

If these types are kept in the Event, they would create hidden obstacles for parsing. We might need them later, but I think it would be better to be as close to the xsd as possible.

Please let me know what you think about it. ; )

@fal05c

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Implement SGTIN EPCs

We need this type of EPC for instance level identifiers.

While it is possible to include class AND object level identifiers (instance: SGTIN, lot: LGTIN: class: GTIN) into an EPCIS EVENT, there should only be one identifier – at the finest available granularity – representing any given object or group objects in the WHAT dimension; for example, if a trade item is serialised, it should be identified in the WHAT dimension by SGTIN.

Original Author: fal05c

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Commented out all of Event.hs

Moved Event related stuff to Event.hs
Added Event.hs to GS1Combinators.cabal
Changed import directive in Main.hs because there's no longer anything left in GS1.hs

Original Author: fal05c

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

CBV-Compatible and CBV-Complaint documents

The section 5 of CBV documentation has identified the standard of the CBV-Complaince and CBV-Comaptibility documents.
I think we can to implement it, after we have the document/event ready. as it contains lots of constraints about the input/output of the document, and the URI.

I will keep this issue opened, and summarise the conditions in the comments as follows.

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Use `Maybe` in `Why`

I am not sure if we need to use Maybe in Why.

I changed the error to MonadError in the why function as well.

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Tests isolation sa

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

parser

use cursors as the basic tool for parsing
the xml must follow the standard, no user vocabulary is allowed
added test xml files

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Bugs to fix after seeing tests fail (3 bugs)

"parse XML to obtain DWhere finds all the dwhere" is failing because the SGLN extension field uses a 0 to represent no extension, but the parser considers the extension to have value of 0

"parse DWhat parses a valid ObjectDWhat" only seems to grab first instance in list of IL, the other 1 expected is in file

"parse DWhat parses a valid AggregationDWhat" also appears to be failing due to a bug in the src

Original Author: zz-per276

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

remove epcistime and exception

Use MonadError to deal with the error handling
Use built-in time whenever possible

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

dispositionValidList

On page 24 of 62 page CBV doc, the third column of the Disposition table is example, it might not be adequate to serve the purpose of validating a (disposition, business step) pair.

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

XML & XSD files for parser implementation and testing

I've copied these out from the documentation provided. The XSD file probably has a few errors from copying from a pdf doc. It's a starting point though.

Original Author: fal05c

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Parser work matt

Original Author: zz-per276

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Implementation of some EPCs

Contains untested implementation of the following EPCs:
ClassLabel
InstanceLabel
Location
SourceDestType
BusinessTransaction

Further contains a minor bugfix in readURILocationEPC:
The larger pattern was being matched first, making the smaller pattern
redundant and hence generating a warning from GHC. It has been taken care of in this Pull Request.

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Standard and User Vocabulary

Distinguish between Standard (reserved) and User-Defined Vocabulary.

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Continuous Integration

Since contributors are actively pushing to the repository, it is important to get CI integrated in the project so that we do not end up pushing something to production that breaks the build.

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Added the types required to create event objects

Wrote some smart constructors for some event objects

Original Author: fal05c

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Refactor labelEPC into classLabelEPC and instanceLabelEPC

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Fixed Tests

The existing tests have been tentatively fixed in this branch.
More tests need to be added eventually.
The tests for Parser are not fixed.

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Cursor API loads the entire XML document into RAM

For the purposes of parsing XML, the Cursor API of the package xml-conduit is being used.

Cursor parses the entire document in one go and loads it all into memory, which means that large documents (we are talking Gigabytes) might cause a problem. In the case of an Operating System error in production environment, we may have to move toward using a streaming API (which loads and parses the document line by line).

It is worth refactoring most of the code in the Parser module so that making that shift is easier.

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Text to String

The codebase converts Text to String a fair number of times, which is not a cheap operation, not to mention - we should not be using strings to begin with.

Unless there is a good reason to, all String occurrences should be Text.

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

xml files from Neil

Original Author: fal05c

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Implement readURI of LabelEPC

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

the usage of Lens should be analysed when Maybe occurs

https://github.csiro.au/zhu020/GS1Combinators/blob/use-lens/src/Data/GS1/DWhy.hs#L24

https://github.csiro.au/Blockchain/GS1Combinators/pull/14#discussion_r3877

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Make vocabularies extensible

We'll need additional vocabulary elements for each deployment. How do we plug them in? Currently we have things like BizStep and Disposition as types, so adding to them isn't possible. Should they be type classes instead?

Original Author: fal05c

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Quantity is not being parsed properly

It is possible that URIs that have a Quantity element to it are not being parsed properly.

The function parseQuantity looks into the very first occurrence of "quantity" and "uom" and returns that.

One possible fix is to go down a particular node in a Cursor and then return a list of Maybe Quantity for that node.

Original Author: ano002

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Extract list of events from parsed XML

Original Author: fal05c

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

add some more implementations

add Object
add URI
add EPC

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

Create constructor functions

create constructor functions for some types start with mk
parse time
generalise some functions in Utils.hs

Original Author: zz-zhu020

(Moved with github-migration-0.1.0.0 (package github-migration-0.1.0.0 revision df9f38b))

data61 / gs1combinators Goto Github PK

gs1combinators's People

Stargazers

Watchers

Forkers

gs1combinators's Issues

Recommend Projects

Recommend Topics

Recommend Org