sri-international/gremlin-visualizer

visualize the graph network corresponding to a gremlin query

Gremlin-Visualizer

This project is to visualize and edit the graph network corresponding to a gremlin query.

Setting Up Gremlin Visualizer

To set up gremlin visualizer, you need to have node.js and npm installed in your system.

• Clone the project

git clone https://github.com/SRI-International/gremlin-visualizer.git

• Install dependencies

npm install

• Run the project

npm start

• Open the browser and navigate to

http://localhost:3000

Note - Frontend starts on port 3000 and simple Node.js server also starts on port 3001. If you need to change the ports, configure in package.json, proxy-server.js, src/constants.js

Setting up with Docker

You can build a Docker image of the gremlin visualizer with the included Dockerfile. This will use the current version of the master branch of the source GitHub repository. The Docker image can be built by calling the docker build command, for example:

docker build --tag=gremlin-visualizer:latest .

The Docker image can then be run by calling docker run and exposing the necessary ports for communication. See Docker's documentation for more options on how to run the image.

docker run --rm -d -p 3000:3000 -p 3001:3001 --name=gremlin-visualizer --add-host=host.docker.internal:host-gateway gremlin-visualizer:latest

Note that --add-host=host.docker.internal:host-gateway is not needed if you don't run your gremlin server in the host machine. If trying to access a gremlin server on localhost, set host.docker.internal in the gremlin-visualizer settings instead.

The Docker container can be stopped by calling docker stop gremlin-visualizer.

Usage

• Start Gremlin-Visualizer as mentioned above
• Start or tunnel a gremlin server
• Specify the host and port of the gremlin server
• Write a gremlin query to retrieve a set of nodes (eg. g.V())

Adding Nodes and Edges

Shift-clicking in empty space will open a dialog to add a new node to the graph.

Shift-click-drag between two nodes will add an edge.

Adding Custom Icons

Add icons into the src/assets/icons folder. Edit src/assets/icons.ts to add a mapping for node type to icon.

Example:

const icons: Record<string, string> = {
  person: require("./icons/person.jpg"),
};

This would render person.jpg for all nodes with the 'person' type label. Capitalization matters.

Custom Node Labels

To configure the initial displayed node labels, edit INITIAL_LABEL_MAPPINGS in src/constants.js. Maps the value of the given node property to the display label of the given node type. Example:

export const INITIAL_LABEL_MAPPINGS = {
  person: 'name'
}

This would set the default display label to the name property on nodes with type 'person'. Capitalization matters.

Changing Graph Visualization Implementation

The graph can be rendered with vis-network, cytoscapejs, or sigmajs. The default is vis-network. To change the graph implementation, edit src/constants.js and change GRAPH_IMPL. The possible values are 'vis', 'cytoscape', or 'sigma'. Available layouts algorithms are specific to each graph implementation.

Features

• If you don't clear the graph and execute another gremlin query, results of previous query and new query will be merged and be shown.
• Node and edge properties are shown once you click on a node/edge
• Change the labels of nodes to any property
• View the set of queries executed to generate the graph
• Traverse in/out from the selected node
• Edit, add, or remove properties to existing nodes and edges
• Add new nodes and edges
• Change layout algorithm
• Save and restore node positions into layout
• Customize node icons and labels base on node types
• Vis-network, cytoscape, and sigma graph rendering engines

gremlin-visualizer's Issues

Investigate why long gremlin queries fail

Running large gremling queries tend to fail with some sort of stack overflow. Investigate if there is a solution to this.

Evaluate Cytoscape and Sigmajs as graph alternatives

UI locks up on Execute and Clear Graph buttons with large graphs

The whole UI locks up while loading or clearing a large graph. Explore using a Webworker to load the graph in the background and keep the UI responsive. Put one of those animated loading circles in the graph while it's loading, and disable the execute and clear graph buttons until the loading is complete. We might also want to disable layouts while loading as well.

Support colors and image loading with sigma and cytoscape backends

These features have not yet been implemented with the cytoscape and sigma graph backends. Nodes should load an image if an image mapping is found in icons.ts. Otherwise, a random color based on node label should be chosen. Nodes of matching label should all have the same color.

Prevent graph relayout

Current Behavior

Currently, we build an entirely new graph each time the nodes, edges, or options change. This results the the graph component reinitializing and causing a new layout.

Expected Behavior

If we run a query that adds an additional node/edge to the graph, or we change a display property other than the layout, we do not want to cause the layout to change. Users will often work to manually layout a graph to improve clarity, and we do not want to disrupt their layout.

There are only 3 times we want to completely reinitialize the graph.

1. Page Load
2. After selecting 'Clear Graph'
3. User applied a new layout to the graph (feature does not exist yet)

Details

To accomplish this, we'll have to modify the getNetwork/getGraph functions to detect differences in the the graph data and selectively apply the changes to the existing graph instead of simply replacing the graph with the a new instance with the latest graph data.

Details view properties table resizing

The column sizes of the properties table resizes when editing a field. The key column should always be taking the minimum space, with the value column taking the remaining space.

Saved Queries Panel

Add a new tab panel for displaying saved queries. Users should be able to run the query right from the panel.

For now, queries will be saved in a map in src/constants.js. The map should be description to gremlin query code. The saved query should display the query descriptions in a list, with an execute button. Displaying the query code is unnecessary, but it might be nice to have a preview panel.

Update documentation

Add file import/export for workspaces

We need to be able to save the workspaces between browser sessions.
Add buttons below the "Save Workspace" and "Load Workspace" buttons to Import/Export worksapces. All workspaces should be imported/exported in one file.

Alternatively, you could extend the proxy-server to save Workspaces to local storage. This would involve adding new endpoints to CREATE/UPDATE/READ/DELETE (CRUD) workspaces, and have the visualizer query the proxy-server instead of browser storage.

Resizable Sidebar

Add the ability to resize the righthand sidebar.

Add controls widget to graph

Add a small control widget with some common controls to the graph.

A good example is the controls widget for react-sigma https://sim51.github.io/react-sigma/docs/example/controls

The controls widget should be highly configurable, and easy to add/remove controls, or have graph specific controls.

I recommend an approach similar to the graph layouts. Export a list specific for each graph, controled by GRAPH_IMPL, that contains an icon and a callback hook. The widget displays each of the icons as buttons and calls the callback hook when pressed.

Basic controls:

• Zoom in
• Zoom out
• Fit/Center graph
• Toggle Physics

Bonus controls:

• Select Layout
• Export graph as png
• Save workspace
• Load workspace

Auto create property fields when existing node/edge type selected

In the new Node/Edge dialog, we should not just suggest the property field names, we should create a field for each known property with the name field already set. Changing the Node/Edge type should reset the field properties the known fields of the new type. Entering in an unknown type should initialize to no fields populated in the dialog box. You should not have to remove the first empty property if you're just creating a type with no properties.

See how gremlify works as an example.

Autosuggest on node labels

Suggest the different known node properties when configuring node labels in the settings component.

Add new Node

Add the ability to add new nodes to the graph. Use gremlify.com for inspiration. Shift-click to create node seems like a good option. This will bring up a modal dialog to enter Vertex Label and add key-value properties. The Node should be immediately viewable in the graph after it is added. The node should be initially located at the location of the shift-click.

Another option besides shift-click is to simply have an add node button in the UI somewhere. Perhaps both options are good.

We want the ability to turn off editing controlled by a property in constants.js. See #6

Add ability to assign custom icons to graph nodes

It should be very easy to assign custom icons to the graph nodes. These assignments should be persistant.

Use Cases:

• Application developer customizes the icons for a particular graph database structure.

• Researcher (non-coder) customizes the graph for a particular experiment or research propsal.

Add ability to delete properties

Currently we cannot delete a property using the UI. We should add the ability to remove a property when viewing an edge/node in the details component.

Suggested Implementation

A small delete button at the end of each edit field in the details list. It needs to work on both edges and nodes. The queries to do each might differ slightly. You may need to use the query-raw proxy endpoint to update the properties correctly.

Add form submission control to adding properties

Ensure that property name and value are required fields, and leave feedback to the user when a field is not filled in.

Add Edges for cytoscape and sigma

Adding edges with Shift-click-drag is only implemented for vis-network graph backend. Implement a similar method of adding edges for cytoscape and sigma graph backends.

Add relationships between nodes

Add an easy and intuitive way to add relationships between nodes. Use gremlify.com as inspiration. Shift-click-drag between two nodes seems like a good implementation. On successful drag from one node to another should bring up a modal dialog creating a relationship from the start node to the target node. The user should be able to add additional key-value properties to the relationship. Saving will send the appropraite gremlin query to add the relationship then show the relationship immediately on the graph.

Another option is adding a button to add relationships. The to-from fields should be auto filled with any selected nodes on the graph, with a button to swap the to-from values.

Avoid overlapping edges

In some states, multiple edges between the same two nodes will be rendered overlapping each other, making it difficult to see each edge. There are ways to avoid this by curving each edge according to the number of edges between the nodes.

Vis

With the vis graph, edges overlap when physics are turned off. When physics are turned off, iterate through edges between two nodes (in both directions) and apply a roundness to them based on the index. E.g. if there are 5 edges between two nodes, give them indexes 0-4, and apply roundess based on the edges index. If there is an odd number of edges, one of them should be straight.

The top answer of this stack overflow question has a great example. Also, the sigma implementation appears to be similar.
https://stackoverflow.com/questions/50766239/avoiding-overlapping-edges-with-minimal-physics-in-vis-js

Cytoscape

Appears to handle it natively without issue.

Sigma

Currently graph does not even support multiple edges. Turn on multi support.

new Graph({multi:true});

Implementation is similar to vis.

https://www.sigmajs.org/storybook/?path=/story/edge-curve--parallel-edges

Refresh node labels not working

Refreshing node labels does not appear to be working.

Steps to reproduce:

• run g.addV('person').property('name', 'bob').property('age', '32') in query editor.
• node with label 'bob' should appear on graph.
• switch to settings and change the field label for node type 'person' from 'name' to 'age'.
• Hit refresh.

Observed:
The label in the graph still shows 'bob'

Expected:
The label in the graph should show '32'

Layout options

Current Behavior

All graphs default to a force-directed graph that continuously applies push/pull forces on connected nodes. When a node is moved, the physics are turned off and node positions are static unless moved by hand or the physics are reenabled.

Desired Behavior

The user should be able to apply different layout algorithms to the graph. The layouts available will depend on the specific graph implementation being used. We will have to dynamically look up which layout options are available based on what has been implemented. We will have to track the current applied layout and isPhysicsEnabled. We don't need an exhaustive list of options. Lets start with force-directed for all 3, and circular for cytoscape and sigma.

Requirements

• Report available layout implementations depending on selected graph.
• Allow user to select desired layout.
• User can start/stop layout with a toggle button. Some layouts, such as circular, run once instead of continuously. Running such a layout should run the layout, then set the isPhysicsEnabled state back to false once it completes.
• Dragging a node should cause any running layout to stop, locking all nodes in place.
• Force-directed layout for vis, cytoscape, and sigma backends.
• Circular for cytoscape and sigma backends.
• Ideally, we have layout controls on the graph, such as in this example https://sim51.github.io/react-sigma/docs/example/controls. However, an entry in the settings tab is acceptable for now.

Example:
Combobox dropdown in settings, populated by available layout types. A toggle button that switches between a run arrow and a stop square depending on isPhysicsEnabled.

Vis graph hierarchical layout

Add hierarchical layout option for vis graph.

Editable properties

Change the node/edge properties list to be editable.

Use Cases:
Researcher wants to update a value in the graph.

Requirements:
There are scenarios where we do not want the user to be able to edit. This entire feature should be configurable with a property in constants.js to turn it off in read only scenarios.

Suggested implementation:
Add a pencil icon to the end of the field which will turn the value label into an input box. The user can edit the field then hit a checkmark icon that has replaced the pencil icon to save changes. This will send the appropriate query to update the field for the appropraite node/edge in the database.

Edges highlight when node is selected

When a node is selected in the graph, all it's connected edges should highlight for better visibility. Some graph backends already do this. Implement this functionality in the remaining graph backends.

Label refresh broken

Labels no longer update during a label refresh. This is due to graphs not being constantly rebuilt on every change. Labels not need to be specifically updated.

Save/restore node positions

We want the user to have the ability to save and restore the node positions of their current layout. We'll name this feature "workspaces" for now to not confuse with graph layout algorithms. Workspace is a remember set of positions for a set of nodes that can be applied to a visualization.

Requirements:

• Save a mapping of id to position of all nodes currently in the visualization.
• Future implmentations will save this to a database or server file store, but for now just download a file or save to session store or local browser storage. Design the API to be easy to change the storage location.
• The user should be able to select a workspace to apply to the current visualization.
• If nodes in the workspace are missing from the current graph visualization, ignore them. They are simply not displayed.
• If nodes are in the visualization, but not in the workspace, then they should be left where they are in the visualization.
• If any physics are active when a workspace is applied, it should be turned off so the nodes do not move from their workspace positions.
• User should be able to overwrite existing workspaces, but the UI should ask for confirmation.

Test with JanusGraph DB

Install janusgraph locally, or use docker to run JanusGraph. Attempt to connect to JanusGraph with the gremlin visualizer, and create/delete/query nodes to verify we are compatible.

Option for persistant node labels

Currently the user can edit node labels once a graph is loaded. We want to be able to preconfigure the node labels for expected graph schemas. Perhaps loading a mapping from constants.js. We should still be able to override the node labels in the UI.

Suggest label and property keys when adding new node

When adding a new node, the form should suggest values for Vertex types already displayed on the graph. You can probably grab values from the nodeLabel keys.

I suggest using the autocomplete component.

When a known type is selected, the form should also auto populate with the known property keys from all vertex's with that label. This might require keeping another list state, or building it up again. We're doing something similar when choosing nodeLabels after a query.

We will want to also do this for edges.

Table View of Data

Add another tab that is a data table view of the graph.

I recommend using collapsible table from https://mui.com/material-ui/react-table/. Show the id, type, and display label in the main table. Show all other properties in the collapsible table. Also consider dense spacing, pagination, and sorting.

Add Unit Tests

We should add unit tests to verify behavior. Target 80% code coverage.

Generic settings store object

Current Behavior

We store the vis-network specific settings object in the Redux store.

Desired behavior

We should store our own abstraction of the settings in the Redux store. Each graph implementation should then keep it's own settings object, and appropriately apply graph specific changes when the Redux store settings updates.

Bonus points if we can also customize the default graph specific settings in constants.js. Or at least make it very easy to change hardcoded defaults. i.e. the graph settings defaults should be specified once, and updates should be selectively made, rather than hardcoding the entire settings value on change.

Do this:

const default settings = {
a: 1,
b: 2,
c: 3
}

function onChange(bState) {
  let newSettings = Object.assign({}, {...settings, b: bState})
  graph.setSettings(newSettings)
}

Not this:

function onChange(bState) {
  graph.setSettings({a: 1, b: bState, c: 3})
}

Details view: add new property

When viewing a Node or Edge in the details view. We want the ability to add a new property that does not yet exist.

Suggested Implementation:
A small '+' button at the bottom of the properties list that will prompt the user for a key and value for the new property. Maybe use the modal dialog from material-ui.