janusgraph/janusgraph-python

JanusGraph Python Gremlin Language Variant (GLV)

JanusGraph is a highly scalable graph database optimized for storing and querying large graphs with billions of vertices and edges distributed across a multi-machine cluster. JanusGraph is a transactional database that can support thousands of concurrent users, complex traversals, and analytic graph queries.

Learn More

The project homepage contains more information on JanusGraph and provides links to documentation, getting-started guides and release downloads.

Visualization

To visualize graphs stored in JanusGraph, you can use any of the following tools:

• Arcade Analytics
• Cytoscape
• Gephi plugin for Apache TinkerPop
• Graphexp
• Graph Explorer
• Graphlytic
• Gremlin-Visualizer
• G.V() - Gremlin IDE
• KeyLines by Cambridge Intelligence
• Ogma by Linkurious
• ReGraph by Cambridge Intelligence
• Tom Sawyer Perspectives

Community

• GitHub Discussions: see GitHub Discussions for all general discussions and questions about JanusGraph

• Discord for interactive discussions and questions about JanusGraph: Join the server

• Stack Overflow: see the janusgraph tag

• Twitter: follow @JanusGraph for news and updates

• LinkedIn: follow JanusGraph for news and updates

• Mailing lists:

  • janusgraph-users (at) lists.lfaidata.foundation (archives) for questions about using JanusGraph, installation, configuration, integrations

    To join with a LF AI & Data account, use the web UI; to subscribe/unsubscribe with an arbitrary email address, send an email to:

    • janusgraph-users+subscribe (at) lists.lfaidata.foundation
    • janusgraph-users+unsubscribe (at) lists.lfaidata.foundation

  • janusgraph-dev (at) lists.lfaidata.foundation (archives) for internal implementation of JanusGraph itself

    To join with a LF AI & Data account, use the web UI; to subscribe/unsubscribe with an arbitrary email address, send an email to:

    • janusgraph-dev+subscribe (at) lists.lfaidata.foundation
    • janusgraph-dev+unsubscribe (at) lists.lfaidata.foundation

  • janusgraph-announce (at) lists.lfaidata.foundation (archives) for new releases and news announcements

    To join with a LF AI & Data account, use the web UI; to subscribe/unsubscribe with an arbitrary email address, send an email to:

    • janusgraph-announce+subscribe (at) lists.lfaidata.foundation
    • janusgraph-announce+unsubscribe (at) lists.lfaidata.foundation

Contributing

Please see CONTRIBUTING.md for more information, including CLAs and best practices for working with GitHub.

Powered by JanusGraph

• Apache Atlas - metadata management for governance (website)
• Eclipse Keti - access control service to protect RESTful APIs (website)
• Exakat - PHP static analysis (website)
• Open Network Automation Platform (ONAP) - automation and orchestration for Software-Defined Networks

• Uber Knowledge Graph (event info)
• Express-Cassandra - Cassandra ORM/ODM/OGM for Node.js with optional support for Elassandra & JanusGraph

• Windup by RedHat - application migration and assessment tool (website)

Users

The following users have deployed JanusGraph in production.

• CELUM
• Crédit Agricole CIB - use case
• eBay - video
• FiNC
• G DATA - blog post series about malware analysis use case
• Netflix - video and slides (graph discussion starts at #86)
• Qihoo 360 (about)
• Red Hat - application migration and assessment tool built on Windup
• Times Internet
• Uber

janusgraph-python's Issues

Create automated scripts to build project and docs

Currently, the library is build using static set of commands like pyb.
We will need to move that to a bash script, so that the library building can happen inside a virtual env, and dependencies are isolated.

Currently, documentation are build by running a set of commands which run to be in succession.

This adds extra level of complexity, as user is not expected to be running each one at a time. To make life easier for end user, a script needs to be developed which would run all these commands one after other.

A .bat file needed to generate docs on Windows.
A .sh file needed to generate docs on NIX systems.

In addition to that, extra features needs to be implemented related to Sphinx cleanup.

1. Sphinx needs to import version, name and other Project metadata from build.py while building docs.
2. Redundant/Optional configuration for Sphinx needs to be removed.

Update version in build.py to reflect new 0.1.0 branch

build.py currently lists the version as 0.0.9 update to 0.1.0 to reflect new 0.1.0 branch

janusgraph-python geoWithin

Looks like the file core/attribute/GeoPredicate/geoWithin.py should be renamed GeoWithin. Doing so allows me to use: from janusgraph_python.structure.io.GraphsonReader import JanusGraphSONReader. Otherwise, I get the error: ModuleNotFoundError: No module named 'janusgraph_python.core.attribute.GeoPredicate.GeoWithin'

Migrate to `testcontainers` from `docker` library for client

testcontainers-python is widely used containerization library used for integration tests. The pros of the library docker which is used currently is that, it provides methods to get the host ip which is very much needed to connect to gremlin-server as part of integration tests which is currently retrieved using Popen which is a security venerability.

Also, dotnet library uses the same package ( testcontainers ) for containerization, and hence for consistency, moving python client to testcontainers-python seems a good idea too.

Add exaustive table of contents on index of docs

The .rst doc for index, source used to render the docs define a list, and we hard code the paths to files. We will need to move that to using toctree which generates automatically ToC for project.

Also, the ToC needs to reference the first page/root page of API docs. Currently it only references User Guide docs.

Move Geoshape to GeoJSON util code as part of PointSerializer and CircleSerializer classes.

Why is this serialization implemented in utils and not in serializer like the other serialization logic? This is a really confusing place. Also the name doesn't hint at the fact that it only applies to GeoShapes.
I would instead create a PointSerializer and a CircleSerializer and put them into the serializer folder.

Originally posted by @FlorianHockmann in https://github.com/_render_node/MDE3OlB1bGxSZXF1ZXN0UmV2aWV3MTkxOTcxOTk1/pull_request_reviews/more_threads

Move to pyenv as virtual env manager

Currently, the library uses virtualenv package to create virtual environments inside which the library is build and installed to global python version.

The drawback with virtualenv is that it expects the Python interpretor to be installed in system and one of following commands work:

• python3
• python3.4
• python3.5
• python3.6

Since it expects Python to be installed, some users might experience Build failures. Also, we don't want users to be manually specifying the python version, as that should be developer knowledge.

To work around that, we are going to use pyenv which doesn't expect you to install a particular version of python, rather it compiles them, and creates an env. This will give us the flexibility of building the lib against each version of python (irrespective of it is being installed or not) and also test it at same time doesn't expect user to test which of above commands work, but still build without fails.

Update docs for building the library

The build scripts for library has undergone a lot of changes over last few days. We changed the environment manager a lot of times. We moved to automatically downloading Python for system, and them removing that too. We updated windows build scripts.

Those changes needs to be documented, and hence the current Docs needs to be updated so that user are updated for the same.

Add serialization support for arrays

After JanusGraph/janusgraph#1295 is implemented, serialization and deserialization of arrays should also be supported in janusgraph-python.

Add support for additional geoshape types

Currently the package supports Point and Circle geometries. Add support for additional geometry types including Box, Line, Polygon, MultiPoint, MultiLineString, MultiPolygon and GeometryCollection.

Update commits to include CLA

Create Integration tests for Client to test for attributes.

Currently only unit tests are developed for the version of client in PR. Based on @FlorianHockmann 's comments on on PR#4.

The Integration tests needs to start a docker container with GraphOfGods loaded, and needs to query into the running docker and check for validity of response.

Fix Codacy errors in 0.1.0 branch

Once the initial PR is merged create a follow-up PR that resolves the issue listed by codacy.

Update JanusGraph client to register custom serializers

Currently when we build a connection to JanusGraph server, the Client registers only the default Serializers and Deserializers which come part of 1.0.0 release. Those are:

1. Point Serializer
2. Circle Serializer
3. GeoShape Deserializer
4. RelationIdentifier Serializer
5. RelationIdentifier Deserializer

We will need to extend the build() method of client to accept JanusGraphReader and JanusGraphWriter arguments if provided to create connection to JanusGraph by adding the passed Serializers/Deserializers.

Since JanusGraphReader and JanusGraphWriter objects take care of registering Serializer and Deserializers, hence the connection to JanusGraph server is not registered against the new serializer/desrializer,

Retrival of coordinates from Gremlin response is reversed.

The response of Gremlin server is of following structure:

{
"type": "Feature",
"geometry": {
"type": "Point",
"coordinates": [125.6, 10.1]
},
"properties": {
"name": "Dinagat Islands"
}
}

Note that in response, the order of Coordinates is reversed. First is Longitude, followed by Latitude. The same logic was reason why #10 issue happened.

To correct that, GeoShape deserializer needs to reverse the order of retrieval of corresponding coordinates elements.

Accordingly unit tests needs to be updated. Resolving this issue closes #10

Update Travis CI to include Miniconda in env

Currently, Travis build use pure python for versions specified to execute all python scripts and build the project.

Since, we are going to start adding Anaconda as pre-requisite for using the library, we will be moving all of builds to start using Anaconda.

The #7 takes care of creating automated script to generate docs, which was earlier done using Pure python evns. Now that is moved to Anaconda.

The Travis build needs to include Anaconda/Miniconda so that build happen using Anaconda.

Also, if possible we will need to add building of docs script to Travis build.

Remove hardcoded credentials for Travis

Remove Hardcoded username and password used currently in Travis CI (.travis.yml) file. They need to be passed as secret env variable.

I'll remove the current instances of all hard coded credentials and do a force re push. Once removed and pushed, @chupman will then take care of setting env vars (from Travis UI) and accordingly update the env vars in travis.yml

The order of Latitude and Longitude in Geo shapes is reversed

The order of Latitude and Longitude is reversed in Geo shapes. First format currently is <Longitude, Latitude> while intuitive way is <Latitude, Longitude>.

We will need to Update GeoShpae classes and also corresponding Unit tests.

Needs to be closed before 1.0.0 release.

Simplify response structure verification from gremlin server in deserializer classes.

Currently, the deserializer class (GeoshapeDeserializer) has very deep level of indentation for checking the validity of response object from Gremlin server. That needs to be simplified.

Based on comment from @mbrukman , and looking at file GeoShapeDeserializer we see very deep level of indendation.

We can modify that into following structure:

if not some_condition():
    raise SomeError

if not some_other_condition():
    raise AnotherError

...

# now, only valid cases remain

Based on following comment

Publish initial version of JanusGraph Python client drivers

The first version of JanusGraph Client drivers shall include the following features:

1: GeoShapes for Point & Circle.
2: GeoPredicate for geoWithin.
3: Test Predicates for textContains, textContainsFuzzy, etc.
4: Edge ID Deserialization.

As further versions are released, more features will be added.