simphony/simphony-osp

A framework that aims to achieve interoperability between software such as simulation engines, databases and data repositories using a knowledge graph as the common language.

SimPhoNy

The SimPhoNy Open Simulation Platform is a framework that aims to achieve interoperability between software such as simulation engines, databases and data repositories using a knowledge graph as the common language. It is focused on the domain of materials science.

SimPhoNy enables:

• Visualization and exploration of OWL ontologies and RDFS Vocabularies
• Wrappers: interfaces between ontologies and software products or digital objects
• Manipulation of ontology-based data: work with ontology individuals, transfer them among different software products using the wrappers, and query the knowledge graph

Installation

SimPhoNy is available on PyPI, so it can be installed using pip

pip install simphony-osp

Detailed installation instructions can be found here.

Documentation

To learn how to use SimPhoNy, check out our documentation, which is located at https://simphony.readthedocs.io/en/v4.0.0/.

In addition, basic usage examples to quickly get started are available in the examples folder.

If you want to build the documentation locally, refer to our documentation repository.

Contributing

If you wish to contribute to SimPhoNy, please read the contributing guidelines.

Acknowledgements

The SimPhoNy Python package originates from the European Project SimPhoNy (Project Nr. 604005). We would like to acknowledge and thank our project partners, especially Enthought, Inc, Centre Internacional de Mètodes Numèrics a l'Enginyeria (CIMNE) and the University of Jyväskylä, for their important contributions to some of the core concepts of SimPhoNy, which were originally demonstrated under the project simphony-common.

simphony-osp's Issues

semantics of remove method

In GitLab by @urbanmatthias on May 7, 2019, 15:20

What p.remove(c) means? Are we deleting c from the dataspace, or are we only detaching it from p?

Remove enum34 from the requirements

In GitLab by @urbanmatthias on Apr 1, 2019, 15:14

enum34 is not future proof, because newer versions of Python do not support it.

emmocuds: Remove newlines from generated annotations

In GitLab by @urbanmatthias on Jun 28, 2019, 16:33

pretty_print to variable

In GitLab by @urbanmatthias on Mar 29, 2019, 14:15

Allow storing the output of the pretty_print function into a variable.
For example:
s = pretty_print(data_space)
or
pretty_print(data_space, s) where s is a variable, in case we just want to have the string in a variable without printing to stdout.

Implement `add()` method

In GitLab by @urbanmatthias on Aug 5, 2019, 11:49

This method should:

• Add the uuid & CUBA key of the element to the Cuds instance
  • Note: This could be a dictionary {uuid: cuba_key} or {cuba_key: [uuid]}
• Add the inverse (uuid and CUBA of self to element added)
• If the session is different:
  • Do a copy of the object
  • Set the session to the session of the new parent
  • Add the object to the registry through the session

Should Cuds.get(rel=SomeRelationship) also consider subclasses of SomeRelationship?

In GitLab by @urbanmatthias on Aug 14, 2019, 10:07

There are two options what Cuds.get(rel=SomeRelationship) could do:

Option 1

Only return the elements connected with SomeRelationship

Option 2

Also return the Elements connected by any subclass of SomeRelationship
e.g. Cuds.get(rel=ActiveRelationship) would return all direct children.

saving direct parent containers

In GitLab by @urbanmatthias on Apr 24, 2019, 14:27

It can be helpful to save the pointers of the directs parent containers of a cuds object.

Currently, the operation p.add(c) only updates p that it now contains c, whereas c is unaware of this change.
Telling c it's directly contained by p makes it possible to efficiently inform p (or more distant ancestors) about future events that may happen to c.

default value of session

In GitLab by @urbanmatthias on Aug 7, 2019, 11:06

In the YML file, the default value of a session should be CoreSession. That is:

SESSION:
    definition:
    parent: CUBA.VALUE
    default: <CoreSession>

However, CoreSession is not a concept in the ontology (nor should it be). Therefore, as a solution, we can use NULL as a default value, that symbolizes internally the CoreSession

cuds2dot fails at import of parser

In GitLab by @urbanmatthias on Feb 23, 2019, 00:58

when running cuds2dot it fails to import parser.

Solution:

• add "." to the import path.

Implement `get()` method

In GitLab by @urbanmatthias on Aug 5, 2019, 16:35

• Decide if all the argument combinations from the docs make sense (get())
• Build a list of queried objects and ask the session for them
  • When querying for uuids, None values could be returned

Adding the current ontology for simopenfoam

In GitLab by @urbanmatthias on Jun 18, 2019, 19:18

Includes the ontology file used to run simopenfoam in the current state. This will be updated in future to be more complaint with the EMMO.

return value of core functions

In GitLab by @urbanmatthias on Aug 7, 2019, 11:14

Since objects are being copied whenever we add them to a new wrapper, it would perhaps be more convenient for a core function to return the new copy rather than the old one.
For example:

a = Atom()
some_sim_wrapper.add(a)
sim_a = some_sim_wrapper.get(a.uid)[0] # here, 'a' is still associated with the CoreSession

Perhaps it would be better to have:

a = Atom()
sim_a = some_sim_wrapper.add(a)

non instantiable attributes as keys

In GitLab by @urbanmatthias on Mar 9, 2019, 00:17

• current behavior: when a cuds is created with, e.g., name or a VALUE, or any leaf entity, this entity does not appear as a key and can be accessed only as a "." attribute.
• desired behavior: also non instantiable attributes appear as keys. so that, e.g.,

c=SomeCuds(name="nameOfCuds")
print c.keys()
> dict_keys([<CUBA.NAME: 'NAME'>])

Replacement of cuds objects with different neighbors. update/add

In GitLab by @urbanmatthias on Aug 22, 2019, 13:59

Assume update or a recursive add causes a cuds object to be replaced.
Let's say new_cuds replaces old_cuds in the session of old_cuds.
There are multiple cases to consider.
Below are the cases and the current behaviour.

• new_cuds has parents, that weren't parents of old_cuds.

  • the parents are already stored in the session of old_cuds.
  • they are not already stored in the session of old_cuds.
    → Add refereces between new_cuds and the parents that are already in the session.
    → Delete references between new_cuds and parents that are not available.

• new_cuds has children, that weren't children of old_cuds.
  → add/update them recursively.

• A parent of old_cuds is no longer a parent of new_cuds.
  → Add a relationship between that parent and the new cuds.
  → Necessary because reference to wrapper-parent should not be deleted on update.

• A child of old_cuds is no longer a child of new_cuds.
  → Remove the relationship between child and new_cuds.

prune() method of CoreSession.

In GitLab by @urbanmatthias on Aug 22, 2019, 12:54

For all wrapper sessions, the wrapper is the only root of the datastructure.
The prune() method usually removes all Elements that are not reachable from the root.

The CoreSession on the other hand can have multiple roots.
It is not obvious which roots to consider when testing for reachability.
It is not obvious, which elements should be deleted from the registry.

We considered the following options:

Option 1

Let the user specify the roots when calling prune.
The interface of the CoreSession.prune() will then be different to prune() of WrapperSession()
This is not very intuitive.

Option 2

Use the garbage_collector.get_referrers() method to check if the user still uses a cuds object.
Unfortunately, the documentation states that this method should only be used for Debugging.

Option 3

Make use of weak references.
Simply converting the registry to a WeakValueDictionary will not work.
Other cuds do not reference their neighbors by strong references.
Instead they are referenced by uid.

Option 3.1

We could maintain a set of uid (copies) of cuds objects, that are referenced by the user.
This set is a WeakSet.
The garbage collector deletes a uid-copy if no reference to that uid-copy exists.
The cuds-objects the user creates and deals with contains a reference to its-uid copy.
In the registry, no references to these copies must be stored.
That way, the garbage collecter will delete the uids in the WeakSet, if the user has no reference to the respective object.
To realize this, the user has to work with objects, that wrap the real cuds objects.
These wrapped objects contain the references to the uid-copies in the weak set.
The registry only stores the real cuds without the references.
--> We can use the uids in the weak set as roots for prune()
Too much effort?

Support for default values in YML file

In GitLab by @urbanmatthias on Jun 24, 2019, 14:44

Add support for the 'default' attribute which is Currently being ignored. Example:

NITROGEN_ATOM:
    definition:
    parent: CUBA.ATOM
    CUBA.NAME:
      default: "N"
      scope: CUBA.SYSTEM
    CUBA.NUMBER:
      default: 7
      scope: CUBA.SYSTEM

CUDS serialisation and standard JSON API

In GitLab by @urbanmatthias on Jan 25, 2019, 18:13

check for example https://jsonapi.org/format/ that optiMADE uses (thanks @casper_andersen for the hint).

How should add() recursively add children?

In GitLab by @urbanmatthias on Aug 8, 2019, 10:10

Assume we call cuds1.add(cuds2) and cuds2.session is different from cuds1.session.
In this case we have to copy cuds2 to the session of cuds1.
Furthermore, the children (e.g. cuds enclosed by cuds2) of cuds2 need to be recursively added to the session of cuds1 as well.
There are multiple ways to do so:

Option 1

We only recursively add the children.
→ Most straightforward to implement

Option 2

We recursively add the children and add relationships to parents, if they are already in cuds1.session.
→ After adding, cuds2 might have different parents in each session.

Option 3

We recursively add children and parents.
→ The user might not expect that the parents are also added.

Option 4

Throw an error if cuds2 has parents which are not in cuds1.session.
→ The user has to add the parents first.

Suggestion: change "reverse" to "inverse" in cuds emmo

In GitLab by @urbanmatthias on Jun 27, 2019, 13:30

emmocuds: Inverse of `encloses` should be `is_enclosed_by`

In GitLab by @urbanmatthias on Jun 28, 2019, 16:33

new types of relationships

In GitLab by @urbanmatthias on May 7, 2019, 16:15

Using only the 'has a' relationship can be restricting and sometimes artificial.
To tackle this, we can add other types of relationships.

The following mechanism has been proposed:

1. There is a 'Relationship' entity in the ontology that all relationships inherit from.
2. The statement p.add(c) should be instead p.add(cuds.classes.HasA().add(c)) if p doesn't already contain a 'HasA' object.
3. The format of the ontology file will be changed as follows:

CUBA.TENSILE_TEST
    definition:
    parent: CUDS.CUDS_ENTITY
    has_a:
        CUDS.SPECIMEN
    performed_at:
        CUDS.LAB    

Should Cuds.get(uid) return a list?

In GitLab by @urbanmatthias on Aug 21, 2019, 14:18

When get() is called with a single uid, it is obvious that only a single element is returned.
The result can either be:

Option 1

The Cuds object with the given uid.
This option is most intuitive.

Option 2

Return a list with a single element.
All other get calls return a list.

Implement `update()` method

In GitLab by @urbanmatthias on Aug 5, 2019, 16:43

Remember that the new version could be missing references to elements that were in the older one. This would leave the other elements with a one way relationship to the newer version (or viceversa).
Option: Make a list of children in the old version and compare it to the one in the newer one. Propagate the changes

simple cuds example for visualisation

In GitLab by @urbanmatthias on Mar 21, 2019, 15:00

Implement a delete by UUID function

In GitLab by @urbanmatthias on Jun 5, 2019, 12:50

Currently, there is no easy way to delete a CUDS object only by its UUID. Rather one has to traverse the whole tree, find all the parents of the element to be deleted and delete it from there.

A function in cuds/utils.py should be implemented, which has the explained above behaviour.

A bug in `cuds/utils/ utils.py` find_cuds() function

In GitLab by @urbanmatthias on Jun 13, 2019, 14:28

There is a recursion bug in the function find_cuds(). The function recurses only one branch of the tree. When it reaches the end of the branch, it returns None and does not traverse the rest of the tree.

Should we define datatypes in the ontology?

In GitLab by @urbanmatthias on Aug 19, 2019, 10:38

To store the cuds objects in the database, we usually need to know the datatypes of the cuds objects' values.

Option 1:

We can define the datatype of the cuds objects in the ontology:

NUMBER:
    definition: ...
    parent: CUBA.VALUE
    datatype: INT

If no datatype is given, we assume it's a string.

Option 2:

We assume each datatype is a string.

cuds2dot should issue error message if graphviz not availabe

In GitLab by @urbanmatthias on Feb 23, 2019, 01:29

null

introduce EMMO branch into SImPhoNy ontology

In GitLab by @urbanmatthias on Feb 24, 2019, 19:46

The EMMO branch existing since long time in a separate git branch that is still python 2.7 based. This issue is to merge it with the master which is python 3 alredady.

querying the ontology

In GitLab by @urbanmatthias on Jun 18, 2019, 15:53

We would like to query the ontology in python, regardless of the data.

Update class_generator to support new ontology format

In GitLab by @urbanmatthias on Jun 11, 2019, 12:44

Read the entities, parents and definitions from the ontology file and the relationships from the schema file

pretty print not printing value

In GitLab by @urbanmatthias on Feb 24, 2019, 19:44

pretty print does not print value of a Determinant/quantitative quantity. Suggestion, add a a check if a value attribute exists and print it if yet.

Default relationship of add(), get(), ...

In GitLab by @urbanmatthias on Aug 22, 2019, 14:12

When we add() without specifying a relationship, we currently use cuds.classes.HasPart as a default.
When we get() without specifying a relationship, we currently return all children (i. e. connected with ActiveRelationship or subclass).

Therefore, we have different default relationships (HasPart vs. ActiveRelationship).
Should we change that?

Update ontology specifications for extended relationship

In GitLab by @urbanmatthias on Jun 3, 2019, 10:17

Define the conventions for supporting multiple relationship types in the YAML cuds ontology file

How to name objects?

In GitLab by @urbanmatthias on Jun 19, 2019, 11:52

There are two options to name objects:

Option 1

>>> m = cuds.classes.Movie(name='Lion King')

(in this case, the class 'CUBA.NAME' is-a (or, inherits from) 'CUBA.VALUE')

Option 2

>>> m = cuds.classes.Movie()
>>> m.add(cuds.classes.Name(value="Lion King"))

(in this case, the class 'CUBA.NAME' has-a 'CUBA.VALUE')

Should we have a convention concerning this?

Suggestion

It can be nice to have something like this:

>>> m = cuds.classes.Movie(desc='Some description')
>>> m.add(cuds.classes.Name(value="Lion King"))

Here 'desc' stands for 'description', and so we keep the first option, but it is clear that in order to name our objects, we need to use the second one.

Remove the name attribute

In GitLab by @urbanmatthias on Mar 18, 2019, 13:34

When an entity requires a name, it will be stated in the ontology. If nothing is said, it will have no name

Redesign the get method

In GitLab by @urbanmatthias on Mar 11, 2019, 13:05

After a discussion with @yoavnash, we (he) came up with a better design for the get method, differentiating between a normal uid input, or one where we provide a CUBA key:

>>> def foo(*uid, cuba_key=None):
...     print('option 1: ' + ','.join([str(a) for a in uid]))
...     print('option 2: ' + str(cuba_key))
...
>>> foo(1,2,3)
option 1: 1,2,3
option 2: None
>>> foo(cuba_key='CUBA.ATOM')
option 1:
option 2: CUBA.ATOM

CUBA-KEY / class confusion

In GitLab by @urbanmatthias on Aug 22, 2019, 14:06

Currently, Cuds.get() has two keyword arguments with the following meaning:

• rel is used to filter by relationship. It expects a subclass of cuds.classes.Relationship.
  get() returns all entities connected with a subclass of the given rel.
• cuba_key is used to filter by entity class. It expects a CUBA-Key.
  get() only returns entities with this cuba_key. No subclasses are considered.

→ Should we unify this to only expect a class or only a cuba-key?
→ Should we unify this to either always consider subclasses or never consider them?

Missing inverse relationship in ontology

In GitLab by @urbanmatthias on Aug 26, 2019, 11:15

Assume the ontology states that class A can have relationship rel with class B.
Assume the entry that class B can have relationship rel^-1 with class A is missing.

There are several options to handle this:

• An exception should be raised.
• The inverse should be added automatically.
• Do not automatically add the inverse when adding a relationship.

Should every relationship have an inverse?

Job Failed #65104

In GitLab by @urbanmatthias on Feb 24, 2019, 19:54

Job #65104 failed for 9fe5460:
fix flake issues

Licenses

In GitLab by @urbanmatthias on Feb 22, 2019, 23:32

The current license is rudimentary, it was meant as temporary one for the transition from SimPhoNy-1.0. Need to fix the license to reflect the contributions from multiple projects (and other partners). need to remove personal names or copy right... The DEB (including @Didrik_Pinte ) should be involved.

Add missing tests

In GitLab by @urbanmatthias on Jul 3, 2019, 15:19

Define unittests for utility functions, parser, json serialisation and other untested functionality

pretty_print h5CUDS proxy objects

In GitLab by @urbanmatthias on Mar 14, 2019, 14:28

though h5cuds implement same interface as cuds, pretty_print of a h5cuds proxy crashes at getting ancestors.
to reproduce:

store a cuds in h5cuds, get it and pretty_print it.

dot representation of a runtime cuds_object

In GitLab by @urbanmatthias on Mar 29, 2019, 14:20

Similar to what we have now with xdot and the ontology files, it would be helpful to have the same visualization for the cuds objects themselves.

Add entity for h5cuds to ontology

In GitLab by @urbanmatthias on Mar 18, 2019, 14:06

to match the behaviour of osp_core, creating an H5Cuds object should set an H5Cuds entity as the root:

h5 = H5Cuds("filename.h5")

We should think about whether this is the expected behaviour for all wrappers (lammps, OF...) or not.

the (fixed, builtin) name of entity as tag not attribute

In GitLab by @urbanmatthias on Mar 10, 2019, 20:55

the name used for an entity is not always the name of the entity, but is meant as a simle user defined id or tag, e.g., the names of a CUBA.PERSON entity can be any thing, like "name of the person Jack", while CUBA.PERSON may have an attribute NAME which in this case JACK, which itself also has a name... confusing.

the name of each CUDS is not an attribute but rather a tag (replacement of an ID).

Suggestion: rename NAME to TAG or OBJECT_TAG to make sure its meant for book keeping and not as a true name attribute of an entity.

Version of `pretty_print`, which returns a raw string

In GitLab by @urbanmatthias on Jun 5, 2019, 09:08

Currently, pretty_print directly outputs to standard output, which although in many cases is desirable, there are certain cases, where the user may want to redirect the output to other destination rather than the standard output (e.g. text file). In this case, returning a pure string representation would be more suitable.

Without breaking the functions current behaviour, one can add an optional parameter return_as_string, which default value would be False

Define __str__ for Cuds

In GitLab by @urbanmatthias on Aug 5, 2019, 11:39

Discuss with @yoavnash what the str(Cuds()) should look like (print own uuid and type, plus children information) and implement it.

print value with pretty print

In GitLab by @urbanmatthias on Mar 20, 2019, 23:51

pretty_print should print unit if exists as well as value

What should SomeRelationship.inverse be?

In GitLab by @urbanmatthias on Aug 6, 2019, 15:09

Let SomeRelationship be a relationship that has the inverse InverseRelationship.
There are two options what SomeRelationship.inverse can be:

Option 1

SomeRelationship.inverse is the class of InverseRelationship

Problem:
To set SomeRelationship.inverse to InverseRelationship, we need to import InverseRelationship in SomeRelationship
To set ReverseRelationship.inverse to SomeRelationship, we need to Import SomeRelationship in InverseRelationship
==> We get circular imports

Option 2

SomeRelationship.inverse is the CUBA-key of InverseRelationship

Problem:
In Cuds.add(*args) we need to get the class of InverseRelationship to add it to the args.

Solution:
Create a mapping from CUBA-key to the corresponding class (e.g. in cuds/classes/generated/cuba_mapping.py)