benmaier/netwulf

Interactive visualization of networks based on Ulf Aslak's d3 web app.

netwulf's Issues

Problem with node attributes (specifically `bipartite')

Netwulf seems to have a problem with some node attributes. It results in not plotting anything.

I have a network that is a monopartite projection of a bipartite network. Therefore every node has the attribute ('bipartite',0).

For using Netwulf I have to delete them manually.

for node in tempToPlot:
   del tempToPlot.node[node]['bipartite']

Find attached two input files, one that works and a second one that doesn't work. (I had to rename them as txt)

It might be only the specific node attribute 'bipartite' or node attributes in general. So if people run into this error, just remove the attributes.

matNetwork_error.txt
matNetwork.txt

Clarify `Statement of need`

as discussed in openjournals/joss-reviews#1425

Statement of need (documentation): A statement of need exists in the documentation, but the focus differs from that highlighted in the generated paper. The documentation states "Netwulf is an interactive visualization tool for networkx Graph-objects, that allows you to produce beautiful looking network visualizations. Simply input a networkx.Graph object, style the network in the interactive console and either download the result as a PNG or pipe the layout back to Python for further processing." This implies the software's focus is on creating visually-pleasing network visualizations easily from networkx objects. However, the paper states "Netwulf is a light-weight Python library that provides a simple API for interactively visualizing a network and returning the computed layout and style. It is build around the philosophy that network manipulation and preprocessing should be done programmatically, but that the efficient generation of a visually appealing network is best done interactively, without code. Therefore, it offers no analysis functionality and only few exploration features, but instead focuses almost entirely on fast and intuitive layout manipulation and node/link styling." This statement implies that the development focus is still in generating visually appealing networks interactively, but is mentions a programmatic interface and lack of analysis tools which obfuscates that meaning somewhat. The two statements are not in direct conflict, but the purpose of the work must be clearly stated and in agreement between the paper and the documentation and should be revised for clarity.

ntwulf.visualize does not accept network as a dict

Although stated in the documentation that the network can be provided as a Dict, it fails with message "Netwulf only supports nx.Graph, nx.DiGraph, and nx.MultiDiGraph".

Object of type 'set' is not JSON serializable

when i try the third example
`import networkx as nx
...: import community
...: from netwulf import visualize
...:
...: G = nx.random_partition_graph([10,10,10],.25,.01)
...: bb = community.best_partition(G) # dict of node-community pairs
...: nx.set_node_attributes(G, bb, 'group')
...:
...:

In [6]: visualize(G)`

Switch to `Pathlib` and abandon `os.path`

as discussed in openjournals/joss-reviews#1425

Additionally, I noticed a lot of use of Python's os library. While this works fine on Linux and Unix devices, it can result in errors on Windows machines. Using Pathlib in place of os would reduce the likelihood of errors on Windows machines. I am not able to verify the functionality of this software on Windows machines (the editor can decide whether this functionality on windows should be formally verified or whether this potential limitation should be listed in readme.md or documentation), but do know that Pathlib would allow the developers to normalize paths for Windows machines easily. Consider opening an issue with the tag enhancement.

See also #19

the config-dict parsed to `visualize` is not respected in the visualization anymore

`Post to Python` does not close the window anymore and also does not stop the server anymore

POST picture back to Python such that it's shown in the Jupyter notebook, if one is used

Post picture data to server

• https://stackoverflow.com/questions/34972072/how-to-send-image-to-server-with-http-post-in-javascript-and-store-base64-in-mon

Display image in jupyter notebook

• https://stackoverflow.com/questions/32370281/how-to-embed-image-or-picture-in-jupyter-notebook-either-from-a-local-machine-o

Find out wether the whole thing is running in a Jupyter notebook

• https://stackoverflow.com/questions/15411967/how-can-i-check-if-code-is-executed-in-the-ipython-notebook

Is netwulf available in anaconda?

I use conda virtual environment, normally install a new library by "conda install -n my_env_name package_name", but netwulf is unavailable now, is there any surrogate approaches to install netwulf into my conda virtual environment?

Format of the CSV Input File

Maybe specify the format of the CSV somewhere and state explicitly that it is an edge list (and not an adjacency matrix or another format.)

A small example network with 5 nodes and 5 (+1) edges is

source,target
node1,node2
node2,node3
node3,node1
node3,node4
node4,node2
node5,node5

(the last entry
entry5, entry5
specifies an isolated node)

The header is case-sensitive.

is netwulf supports directed networkx ?

hi!
how to add arrows and weights to graph to visualize ?

Add community guidelines

Hi @ulfaslak @benmaier, are there community guidelines for this project? For example, a contributing guide (usually CONTRIBUTING.md) and a code of conduct (usually CODE_OF_CONDUCT.md from Contributor Covenant). I couldn't find anything in the repository or the documentation.

These are required for the JOSS submission (openjournals/joss-reviews#1425) and are usually straight forward to add. So I though I'd give you a head start before the review starts 🙂

nodes disappear when clicking on the nodes on a touchscreen

I was viewing the netwulf viz on a touchscreen laptop monitor, and if I try to move the nodes around by touching the screen, the nodes disappear.

Is there any way around this?

Canvas size ,self-loops ,large graphs?

1.Is there a way to change the canvas size?
2.Is there a way to display self-loops?
3.Are visualizations of 12.000.000+ edges supported in freeze mode?

Nice and easy package to use though..great work!
Thanks!

color difference between canvas and download/reproduction

So here's a weird one: While playing around with colors and reproducibility, I've noticed that my matplotlib-figures looked brighter than what I've seen on the screen. And apparently that's true, even though the downloaded figure has the right colors again:

funnily enough, the grey node stroke did not change between display in the browser and matplotlib/downloaded png. Ideas what's happening here?

Show edge labels

Hi,

I've tried to reproduce this example, however, the interactive viz. doesn't seem to support this feature (using netwulf v0.1.5). Could you please help?

KeyboardInterrupt starts expensive background process

This is a weird one. When I KeyboardInterrupt in a Jupyter Notebook, a very CPU hungry Python process starts.

When I restart the kernel, it terminates. If I terminate it in the activity monitor, the Jupyter kernel becomes busy (no output in terminal).

Even weirder, if I launch a new visualization, IT STOPS.

bugs in documentation code

as remarked by @vc1492a in openjournals/joss-reviews#1425

I noticed one small item that could be addressed in the examples. In the below screenshot, it appears that a necessary matplotlib import is missing:

I believe this should include import matplotlib.pyplot as plt.

Additionally, in the example located in the filtering section, grp = {u: 'ABCDE'[u%5] for u in G.nodes()} and nx.set_node_attributes(G, grp, 'wum') should be executed after the first visualization and before the second visualization, as opposed to before the first. Executing those lines prior to the first example results in the following error when opening the visualization interface:

When posting back to python, close the window and stop the server

Getting Dataset does not have a key 'nodes'

Hello,

I have build a graph network using networkx and it works well as follows:

G = nx.from_pandas_edgelist(results.head(100), 'source', 'target', edge_attr=True, create_using = nx.MultiDiGraph)
G.nodes
nx.draw(G, with_labels=True, pos = nx.spring_layout(G), alpha = 0.8)

When i then call visualize(G), i get Dataset does not have a key 'nodes'.

Can you help on this please?

Thanks

netwulf visualize ignores edge colors

Hi,

When I create a graph from pandas as:
G = nx.from_pandas_edgelist(df, "node1", "node2", edge_attr=['weight', 'color'])

Properties are correctly set, e.g.:
('node1', 'node2', {'weight': 0.01, 'color': 'red'})

However, when running:
network, config = visualize(G, port=port)

the weight attribute is correctly visualized, but the color one is not (all edges are grey).

How can I get around this issue?

Thanks,
Davide

Highlight connected nodes and edges

Hello!

Can I propose a new feature where you click a node and highlights the connected nodes and edges? This will easily show the directly related nodes in a large network.

Thank you

Posting picture back to Python should return the figure object

Currently, when using the "Post to Python" feature, visualize returns two things: (1) the node-link dictionary with node positions and (2) the config dictionary. As a third tuple element it should also return the figure object, then one can handle that however one likes.

visualize(network) should let the user specify which node and edge attributes to use

I have networks whose nodes may be grouped and therefore coloured differently according to several distinct node attributes. Likewise, my networks hold several different attributes inside one edge (total count, percentage) and I need to use either of them, depending on the situation, to layout the network.

I propose to allow the user to pass two strings to visualize, one to decide which node attribute to use as the denominator for groups, the other to specify which numerical edge attribute to use as the weight:

visualize(network, node_group='node_attr_1', edge_weight='edge_attr_2') as a specific example.

I will open a pull request to this end.

When nodes are dragged, labels stay on for that node

Not sure if I am interacting with netwulf incorrectly, but after running the following code,

import networkx as nx
from netwulf import visualize

G = nx.barabasi_albert_graph(100,2)

visualize(G)

with "Display labels" off, when I drag a node in the visualization, the label stays on the node, even if I click it again. If I don't drag the node, then I can toggle the label on and off just fine. The only way to hide the label on that node again is to turn on the "Display labels" and then turn it off again.

Very minimal importance, but it was unexpected behavior, so I figured I would bring it up.

Great project!

Make compatible for Windows and test on Windows

anybody got one of those PCs?

Post to python functionality not working

I am not able to get the Post to Python functionality working in my environment. More specifically, the UI indicates success and closes the browser window, but the Python kernel remains busy indefinitely and on manually stopping execution, the network_properties object is not returned. This occurred in both a Jupyter notebook as well as a Python session session within a terminal window.

Environment details:

• Python 3.5.2
• netwulf 0.0.14
• Jupyter 4.3.0
• macOS 10.14.4

label position examples are deprecated in the docs

they should use x_canvas and y_canvas and also y should be flipped with height

how to make .exe in Windows with pyinstaller and netwulf included [SOLVED]

Hi !
While packing netwulf in windows with pyinstaller myscript.py -D i got errors.
The first one was that matplotlib wasn't work correctly so i downgraded it to matplotlib==3.0.3
Than i fixed mathplotlib in its dir '/site-packages/matplotlib/mpl-data/matplotlibrc': TkAgg to Agg following this issue:
https://stackoverflow.com/questions/20582384/importerror-no-module-named-backend-tkagg
Than i packed my script again pyinstaller myscript.py -D and had error: distutils.errors.DistutilsFileError: cannot copy tree ...
So i copied dir netwulf/js from native package to my package dist dir: dist\myscript\netwulf\js

After that my script.exe works correctly using netwulf package!

Reproducibility of a previously stylized network not given anymore

The behavior of reproducing a stylized network should be:

1. Open a network visualization
2. stylize network
3. retrieve stylized network and configs
4. bind retrieved positions to network
5. start new visualization with bound positions and config
6. the new visualization should be exactly the same as the old visualization

I've checked out an old version of the javascript base, namely the following commit benmaier/network_styling_with_d3@86762e2

Then I ran the following code

from netwulf.tools import bind_positions_to_network
import networkx as nx
import pprint
pp = pprint.PrettyPrinter(indent=4)

G = nx.Graph()
G.add_nodes_from(range(10))
G.add_nodes_from("abcde")
G.add_edges_from([("a","b")])

from netwulf import visualize
props, config = visualize(G)
config['Zoom'] = 0.666

pp.pprint(props)
pp.pprint(config)
bind_positions_to_network(G, props)
visualize(G, config=config)

this produces the first vis:

and then the second vis:

Afterwards I've checked out the following commit of the javascript base, which is this: benmaier/network_styling_with_d3@a88f4d3

Now the first vis is

and the second vis is

which is different, but it shouldn't be, right?

I think the part of the code which is responsible for this is the part I commented here:

benmaier/network_styling_with_d3@a88f4d3#r33573089

Add the required Python versions to setup.py

It's best to specify the compatible Python versions in the setup.py using the python_requires argument. This let's pip et al know if your Python version is compatible and avoids installing broken packages.

It will probably be useful if adopting Pathlib which has some features that are 3.6+ only. I don't see many reasons to use 3.5 since most of the scientific stack has moved on to 3.6.

openjournals/joss-reviews#1425

Adding changes in CHANGELOG.md

I've noticed that for v0.2 several changes were made that have not been logged in CHANGELOG.md. So far I've found:

• the node attribute radius is now called size. Ulf, could you explain why you chose to rename this? I like radius because I know what it means because it has a geometric defnition. size is rather ambiguous.
• when redrawing with matplotlib, overlapping nodes are joined such that strokes around nodes do not show anymore. was that intended? I think it makes the visualizations kind of awkward looking.

Post back to Python

Sometimes you don't want a png, you want properties of the network like position and size of nodes. We should add a button which uses POST to transfer a json with all network properties back to python. Then you can easily replicate the work you put in, in whatever format.

write short Sphinx documentation

Add changelog

see openjournals/joss-reviews#1425

consider adding a changelog.md file that details changes, additions, and fixes made to the code with each release version. Lastly, it seems that semantic versioning is used but this isn't make explicitly clear in readme.md - not that it needs to be, but adding details such as these, along with a changelog.md, would go a long way towards easing the entry into development for those which may want to contribute to the package.

Save visualization

Hi,

I am testing Netwulf on a project and I would like to save the webpage. If I save the page in Chrome it and when re-open it, no graph appears and the UI is frozen.

Is there a proper way to save this from python?

Thank you!
Cheers
Claudio

EDIT: I also tied to open it from ~/.netwulf directly, or through a Chrome web server, but still the same result.

visualization with initially given positions is unfrozen when node properties are changed

Automated testing

openjournals/joss-reviews#1425

Automated tests: I did not see any automated tests in this repository that could be used to verify the functionality of the software. Revisions of this work should include a series of unit tests that can be used to verify the functionality of the software, through testing of function outputs, input and output formatting, etc. There are a variety of ways to write and use unit tests in Python, two of which are the pytest and pytest-cov packages - the former allows developers to execute unit tests (with helpful extensions for writing tests, too) while the latter provides functionality to pytest for coverage statistics, e.g. which lines of code your unit tests cover and the percentage of lines in each file (and overall, the package) that is covered by the written unit tests (this coverage statistic could be reported in readme.md). Additionally, if you're looking to have others contribute to the project more easily in the future consider setting up a continuous integration pipeline that will automatically run tests and report the test results (e.g. using Travis CI) and coverage (e.g. using Coveralls) with any pushes to master or dev and for any pull requests opened by open-source contributors. This pipeline would ease the burden on making sure new additions and/or changes to software work as intended, and would also allow maintainers to formally test against every version of Python 3 that should be supported. This would allow the authors to state "this software works in Python versions 3.5 - 3.7" as opposed to simply stating "works in Python 3".

Visualizing in google collab

Hi,

First of all thank you for creating such a great tool. I've recently started using netwulf and it works fine on jupyter notebook. However, I'm unable to visualize the network when I run the same code in Google collab. Would anyone know why and what I can do to fix that?

Netwulf changes the current working directory

After running the visualize function, I found that os.getcwd() returned /home/<user>/.netwulf.

Is this to be expected?

Can't post visualization back to python

Hi developers,

i am using your cool tool to visualize a network. But i don't have the option to post my visualization back to python as you have in your tutorial on https://netwulf.readthedocs.io/en/latest/python_api/post_back.html Instead i have the option to upload or save a json. Screenshot of my display

I have tried to reinstall your package but it still only gives me the option to upload or save a json file.
What am i doing wrong?

kind regards

some questions

Great thanks for your program! I look at it after d3.js, networkx.
But have some questions:
1 how to change canvas in code (width & height) ?
2 how to load edgelist to make a layered network (like in webweb) ?
3 how to add weights to show like in d3.js, networkx ?
(labels = nx.get_edge_attributes(G,'weight')
nx.draw_networkx_edge_labels(G,pos,edge_labels=labels))
4 is it possible to switch on edges i want to watch the path from one node to another ?
5 how to find nodes (edges) names like in webweb and hightlight them ?
6 is is possible to make undruggable nodes that i touched not freezing all nodes at once ?

Let `visualize` be run without argument such that the interface opens and a file can be loaded manually

as suggested by @vc1492a in openjournals/joss-reviews#1425

Some potentially useful functionality would be to allow a user to start the visualization interface without the need to pass a graph object into visualize(). This may be useful in the case where the user has a file that they wish to examine and want to upload, but may not want to load first within the Python environment. By making the graph object an optional argument, the user could start the visualization interface and then explore one or more files through the upload functionality. This would be a nice feature in future releases but isn't necessary to address as part of this review. Consider opening an issue with the tag enhancement.

Colouring nodes by property (e.g., degree)

This is not really a bug but rather a workaround that I wanted to make available for others (and maybe you want to integrate it at some point into the library itself).

I wanted to color all nodes by a node property. It is possible to achieve this by using the coloring by `group'. One only has to assign each node an html color according to its value.

Below I give a minimal example that colors a Barabasi-Albert network by node degree.

The result is this image (where nodes are also sized proportionally to their degree). I choose the RGB color to be (X,0,20), such that it is from dark blue to red but this can be adapted as needed. Probably one could also automatically use an existing colormap.

G = nx.barabasi_albert_graph(100,1) # construct a BA graph
deg = [val for (node, val) in G.degree()] # compute its degree

# prepare
degreeColor =[]
valueToPlot = np.log10(deg)
# we rescale the colours to be in the RGB format (0 to 255 for three colours)
valueToPlotRescaled = 255*(valueToPlot - np.min(valueToPlot))/np.max(valueToPlot)

for i in valueToPlotRescaled:
    color = '#%02x%02x%02x' % (int(i), 0, 50) # here we use (X,0,50) as RGB with X beign the log(degree) for eahc node
    degreeColor.append(color)
# zip it up into a dictionary and set it as node attribute
dictionaryColor = dict(zip(list(G.nodes), degreeColor))
nx.set_node_attributes(G, dictionaryColor,"group")

# The actual illustration
network, config = wulf.visualize(G, plot_in_cell_below=False)
fig, ax = wulf.draw_netwulf(network)
plt.savefig("BAnetworkWithDegreeColoured.pdf")

Let me know if there is an easier way to achieve this; anyways, I thought that could be useful for other users.