distributedclocks/tsviz

Tool to visualize executions of multi-threaded systems

tsviz's Issues

Time scale selector should be modified

The current time scale selector consists of a text field in which the user can input a value to be used as the minimum distance between two consecutive nodes on the graph, and a button to refresh the graph.

Suggestion: Implement the time scale selector as a slider, placing it under the search bar to make use of the blank space. The slider values should be calculated automatically, and range from a complete "Zoom Out" option, in which the user could see the whole log in one screen (without having to scroll), and a complete "Zoom In" option, in which the smallest time difference between two events in the log (aside from zero) would be scale to the smallest time difference in screen.

[Issue created by mhnnunes: 2016-08-11]

Search results chart scaling buttons should be out of charts tab

Currently the buttons to change the search results charts scale are scrolling along with the charts, which makes it difficult for the user to change the charts scale at any point in the chart. These buttons should be independent of the charts, and fixed in the sidebar.

[Issue created by mhnnunes: 2016-08-11]
[Last updated on bitbucket: 2016-08-22]

[Comment created by harjeetlalh: 2016-08-22]
Fixes issue #21.

→ <<cset 1c40384>>

Text overlay in sidebar

When the nodes have long texts as their descriptions, this text overlays with other information on the sidebar.

[Issue created by mhnnunes: 2016-11-20]
[Last updated on bitbucket: 2017-06-04]

[Comment created by mhnnunes: 2017-06-04]
Fix issue #38

→ <<cset 8218104>>

Numbers shown in compressed region are slightly off

The number shown in a compressed region is slightly different from the one calculated when the user clicks on the nodes that delimit this region:

[Issue created by mhnnunes: 2016-08-10]
[Last updated on bitbucket: 2016-08-12]

[Comment created by mhnnunes: 2016-08-12]
Fixed in commit << a71be19 >>

Node info on sidebar is not selectable

When the user clicks on a node, information about this node is displayed on the sidebar. However, when the user tries to select something, like copying out a timestamp for example, this information disappears.

[Issue created by mhnnunes: 2016-08-15]
[Last updated on bitbucket: 2016-08-22]

[Comment created by harjeetlalh: 2016-08-22]
Fixes issue #24.

→ <<cset 8f9754b>>

Node positioning should be reviewed

With the presence of the ruler, guides and the time difference display on the sidebar, it became visible that a few nodes might be positioned a little higher or a little lower than they should. So the ruler or the node positioning should be reviewed.

Edit: maybe the problem can be fixed by changing the part in js/model/modelGraph.js in which the minimum and maximum timestamps are saved.

[Issue created by mhnnunes: 2016-06-27]
[Last updated on bitbucket: 2016-08-09]

[Comment created by harjeetlalh: 2016-08-08]
I think you've fixed this already

[Comment created by mhnnunes: 2016-08-09]
Yes, indeed.

After hovering off nodes, they do not return to their original size

This is similar to the edge hover highlight issue where the highlight would stick even after mouse off. In this case, the increased size sticks.

[Issue created by harjeetlalh: 2016-09-19]
[Last updated on bitbucket: 2016-09-28]

[Comment created by ashayas: 2016-09-28]
Fixed issue #32: reset node size on hover off

→ <>

Elapsed time info disappears on graph transformation

When the user expands a collapsed node, or hides a thread, the "Elapsed time" information, displayed on the compressed intervals, disappears.

The first picture represents the graph's compressed interval on it's initial state, the second picture represents how it is left after the transformation.

[Issue created by mhnnunes: 2017-03-01]
[Last updated on bitbucket: 2017-03-05]

[Comment created by mhnnunes: 2017-03-05]
Fixed on commit 0324e4f

Timestamp format should be reviewed

The timestamps are currently saved in strings. But they are in a format that could be translated into a Date value. Right now we are converting them into Numbers to make operations with them but they are very large numbers and they overflow Javascript's Number format, so the results are not accurate.

E.g. in the picture below, the correct result would be 1100:

If the timestamps are generated by counting nanoseconds, i.e. it represents absolute values of nanoseconds, calculating time differences by subtracting Numbers is the right way to go. But if it's generated by date encoding, we should change it so that we perform operations on Date variables.

Action: Check with Prof. Sasha how the timestamps are generated and see if it's best to save it in a Date variable or if we can just operate in those absolute Numbers that are stored in strings.

[Issue created by mhnnunes: 2016-06-08]
[Last updated on bitbucket: 2017-04-17]

[Comment created by grahamstl: 2017-04-17]
In the current version (03f7ab9), the string timestamps are being truncated at the front (removing the first three characters of each) before doing the math in order to generate more accurate results (see the ModelGraph constructor). But if the timestamp format produces timestamps that are too short, this will not work.

The documentation simply says 'physical clock timestamp' but gives no indication for the format or units (at least not on the main page), so there is the possibility that they will use a timestamping procedure that causes this bug to appear. Ideally, our new format will take this into consideration.

Space in time skip regions should be used to show time span

Currently the time skip regions are blank. These must be used to show the amount of time elapsed within that region.

[Issue created by bestchai: 2016-08-04]
[Last updated on bitbucket: 2016-08-12]

[Comment created by mhnnunes: 2016-08-12]
Fixed in commit << a71be19 >>

Node/edge info display on sidebar is inconsistent

When the user clicks on an edge between two nodes, this is the format of the information displayed on the sidebar:

However, when clicking on the nodes separately, the information is displayed like this:

These formats should be consistent.

[Issue created by mhnnunes: 2016-08-11]
[Last updated on bitbucket: 2016-08-22]

[Comment created by harjeetlalh: 2016-08-22]
Fixes issue #22.

→ <<cset 2fc8a4f>>

Small Time Frames

Rendering the graph on smaller timescales (typically <= 100ns on normal browsers and logs as shown) will cause chrome to run out of memory and crash the program.

[Issue created by ashayas: 2016-11-03]
[Last updated on bitbucket: 2016-11-04]

[Comment created by ashayas: 2016-11-04]
One issue is that this is ram dependent but I think if we can assume a more modest ram size for the user and determine thresholds and give this error accordingly it would be best. I will figure out a way that works best.

[Comment created by bestchai: 2016-11-04]
Can you detect when this is a problem (e.g., when number of events is above some limit)? in this case you can show an error to the user telling them that they should select a larger granularity value, or a value above 100 (the more precise the better).

"Clear" button from search bar is not being drawn in the right place

The "clear" button from the search bar should be drawn inside of the input area, but instead it's being drawn over the number of results from the search.

[Issue created by mhnnunes: 2016-06-08]
[Last updated on bitbucket: 2016-06-22]

[Comment created by mhnnunes: 2016-06-22]
Bug fixed in commit c6b2f1f5281099c971c2a964001ca0aad3fdf4a7

First slice of ruler goes past blank space

When there are compressed intervals in the graph, the first slice of the ruler goes sligthly past the start of the first interval.

The other compressed intervals are correct.

[Issue created by mhnnunes: 2016-08-09]

[Comment created by mhnnunes: 2016-08-09]
Fixed on << e521f8f >>

Showing a busy indicator when log is being processed

Processing a long log takes time. It's important to signal to the user that the tool is actively working, and is not stuck. The simplest solution is to (1) place an overlay over the input screen so that it is seen as disabled, and (2) use a busy indicator that animates a simple shape until the log is processed. Here is an example of a gif we can use:

[Issue created by bestchai: 2016-08-05]

Node information not displayed in multi-execution mode

Sometimes, in multi-execution mode, the dotted line highlighting the node position is not shown. And sometimes, the time difference between two selected nodes is also not shown.

[Issue created by mhnnunes: 2017-06-03]

Ruler continues way past the last event

For the log fslock_24t_4sp.log I'm seeing the ruler extend way past the last event, showing about 3-4 empty screens of space. The diagram should be tight and fit to the last event, and no more.

[Issue created by bestchai: 2016-08-04]
[Last updated on bitbucket: 2016-08-09]

[Comment created by mhnnunes: 2016-08-09]
Fixed on << 0634360734e >>

Host bar stops scrolling horizontally after a graph transformation occurs

Initially the host bar scrolls horizontally along with the hosts timelines. But after some graph transformation is applied, this scrolling stops working and the hosts timelines scroll by themselves.

Edit: Bug present in the original ShiViz

Edit 2: fixed for most transformations, only happening after clearing search results now

[Issue created by mhnnunes: 2016-06-10]
[Last updated on bitbucket: 2016-06-22]

[Comment created by mhnnunes: 2016-06-22]
Bug fixed in commit c6b2f1f5281099c971c2a964001ca0aad3fdf4a7

Timestamp should always be relative to trace

Timestamp of a node should always be relative to the start of the trace

[Issue created by ashayas: 2016-10-12]
[Last updated on bitbucket: 2016-10-22]

[Comment created by ashayas: 2016-10-22]
Resolved in commit e236975

Ruler should be adjusted in time skip regions

The ruler currently does not adjust for time skipped in a skip region. The ruler must show that that is a skip and must accurately reflect the time skipped.

[Issue created by bestchai: 2016-08-04]
[Last updated on bitbucket: 2016-08-09]

[Comment created by mhnnunes: 2016-08-09]
fixed on << dccf023996 >>

Thread lines are too thick

Compare original shiviz:

and shiviz++:

Lines in shiviz++ are too thick, drawing user's attention to them unnecessarily.

[Issue created by bestchai: 2016-08-04]
[Last updated on bitbucket: 2016-08-09]

[Comment created by harjeetlalh: 2016-08-09]
Fixes issue #10.

→ <>

[Comment created by harjeetlalh: 2016-08-09]
Made edge stroke-width set to 1. Addresses issue #10

→ <<cset 8a5e684>>

Hover off edges keeps color

When a user hovers over an edge it displays the time between edges and highlights the node. When we hover off it keeps the edge highlighted.

[Issue created by ashayas: 2016-09-18]

[Comment created by ashayas: 2016-09-18]
Fixed, tested and merged into master.

Edges between two nodes containing collapsed events are not drawn

When the user selects a coarse timescale, the graph eventually collapses nodes that are connected. When these nodes are collapsed, edges between them are not drawn. The graph then looks like a bunch of collapsed nodes with no edges between them, giving the user no little to no information about the log.

[Issue created by mhnnunes: 2016-06-22]
[Last updated on bitbucket: 2016-08-09]

Slider bar overlays over thread number

When you hover over a thread box it displays which thread it is. But when this is done under the slider it doesn't show the thread.

[Issue created by ashayas: 2016-09-18]
[Last updated on bitbucket: 2016-10-04]

[Comment created by ashayas: 2016-10-04]
Fixed issue #9 and #30

→ <>

Redundant tick marks after threshold

After the maximum number of digits that can be fit in the ruler's space is hit, the tick marks get redundant.

[Issue created by mhnnunes: 2016-08-10]

Host colors are not always unique

Currently, ShiViz is generating colors for hosts using d3.scale.category20(), which gives us only 20 unique colors. If the number of hosts is greater than that, the user will see two hosts using the same color.

[Issue created by mhnnunes: 2016-06-22]
[Last updated on bitbucket: 2017-04-22]

[Comment created by grahamstl: 2017-04-22]
The new colours should be darker, rather than lighter; and perhaps we should remove some of the less readable colours in the process of fixing it.

Search window unresponsive

When I try to do the search for the first time, it works okay. If I later click on the search window to change the parameters of the search, the window becomes unresponsive for a very long time as if it is trying to process the entire data even though all I did was click inside the search box.

[Issue created by sasha_f: 2016-12-15]
[Last updated on bitbucket: 2017-01-01]

[Comment created by ashayas: 2017-01-01]
Resolved issue #41

→ <<cset 4254999>>

Number of search results is not shown

In ShiViz, when the user searches for a term, the number of results is shown next to the search bar, as shown below:

In TSViz, this is not shown.

[Issue created by mhnnunes: 2017-06-03]
[Last updated on bitbucket: 2017-06-04]

[Comment created by mhnnunes: 2017-06-04]
Fix issue #49

→ <>

Edges with no event at top cannot be selected/do not show time

Currently one can only interact with lines between two events. But, we should enable this interaction for lines that have no event at the start, and just count from 0 for the entire trace. So, it should be possible to select the edge above the first grey thread event below:

[Issue created by bestchai: 2016-08-04]
[Last updated on bitbucket: 2016-08-22]

[Comment created by bestchai: 2016-08-09]
The edge can be clicked, but it's not computing the right time value -- I'm only seeing 0 as the time delta for that edge. The value should be simply the time at which the event occurred (on the edge being clicked) relative to the very first event in the log

[Comment created by bestchai: 2016-08-09]
re-opening because time is not being computed properly

[Comment created by harjeetlalh: 2016-08-08]
This fixes issue #11 also handles the edge click.

→ <<cset 3539b5b>>

[Comment created by harjeetlalh: 2016-08-08]
Fixes issue #11

→ <<cset 3ec29b4>>

[Comment created by harjeetlalh: 2016-08-08]
Wait forgot to handle edge click.

[Comment created by harjeetlalh: 2016-08-22]
Fixes issue #11.

→ <<cset 97123ce>>

Node collapsing should be reviewed

When collapsing nodes based on time proximity, some nodes are not collapsed:

Review why that happens.

[Issue created by mhnnunes: 2016-08-11]

Giving an alert on backspace

Right now users may accidentally hit backspace which would take them back to the homepage. There should be a warning before they do this.

[Issue created by ashayas: 2016-10-12]
[Last updated on bitbucket: 2016-10-13]

[Comment created by ashayas: 2016-10-13]
Fixed issue #33

→ <>

Rework TSViz Wiki home page.

Currently the wiki home page is just an adaptation of the ShiViz wiki home page. It needs to show TSViz images and describe TSViz features.

[Issue created by grahamstl: 2017-05-06]

Selecting node in search should select node on graph

If we select a node on the search results graphs it should highlight on the graph on the left. An example is seen in the following image where the blue node on the left should be selected.

[Issue created by ashayas: 2016-09-18]
[Last updated on bitbucket: 2016-09-28]

[Comment created by ashayas: 2016-09-28]
Resolved

[Comment created by harjeetlalh: 2016-09-18]
I think this is partly caused by the visualization being resized.

No search results indicator

When no search results are found we need to display some indication of that and also take care of the state of the search bar and sidebar.

[Issue created by harjeetlalh: 2016-09-18]

Slider underneath searchbar is too long after the sidebar resizes

The slider should resize with the sidebar so the information in the sidebar doesn't get hidden from view.

[Issue created by harjeetlalh: 2016-09-18]

Examples break in off-line mode in Chrome

Chrome has a restrictive security policy that disallows local file access from a page. This breaks our example logs when the user is offline in Chrome (Safari works).

Chrome error that I get is:

jquery-3.1.0.min.js:4 XMLHttpRequest cannot load file:///Users/ivan/repos/tsviz/log/fslock_24t_4sp.log. Cross origin requests are only supported for protocol schemes: http, data, chrome, chrome-extension, https, chrome-extension-resource.

A fix to this is (1) detect that an error of this type has been thrown, and (2) recommend to the user that they restart chrome disabling web security so that the example log could be loaded:
http://stackoverflow.com/questions/4819060/allow-google-chrome-to-use-xmlhttprequest-to-load-a-url-from-a-local-file

(The same problem exists in ShiViz; to close this issue the problem must be solved in both)

[Issue created by bestchai: 2016-11-26]
[Last updated on bitbucket: 2016-12-02]

Sidebar should be reviewed and organized

Lots of improvements are necessary to the sidebar. Some of them are listed below:

1. Organize information into tabs, some examples of tabs are: Node info tab, search results tab, graph options tab.

2. Sidebar should be vertically scrollable, on small screens, part of it's content gets hidden. As shown in the image below, graph options are out of the screen, and even the bottom of the search results chart gets cut off.

3. Sidebar should resize on window resize: the sidebar has a fixed size in pixels. So on large screens it occupies the same space as always. It should be optimized to recognize blank space between the screen's and the graph's right corners and make use of that space.

[Issue created by mhnnunes: 2016-08-11]
[Last updated on bitbucket: 2016-08-30]

[Comment created by mhnnunes: 2016-08-24]
Earlier, the overflow attribute on the sidebar div's css was set to hidden. On commit 4fc7181 I changed it to "overflow-y: auto;", enabling vertical scrolling.

[Comment created by harjeetlalh: 2016-08-24]
I think the sidebar scrolling up and down when there is not enough space does work currently. Not sure why it may have not been working earlier.

[Comment created by harjeetlalh: 2016-08-30]
Sidebar automatically resizes upon resizing the window. Addresses issue #20.

→ <<cset 0180b40>>

Thickness of lines should stay consistent

On search some edges are bolded more than others: this should stay consistent

[Issue created by ashayas: 2016-10-12]
[Last updated on bitbucket: 2017-01-01]

[Comment created by ashayas: 2017-01-01]
Log events matching search results have uniform boldness and those not related to the search are lighter.

Edge frequency stats

Clicking on an edge should list not just the latency between the events for the edge and their details, but also statistics about the distribution of edges that are similar to this edge.

Some questions to answer:

• What edges are considered similar? e.g., could be 'all edged between these two thread', or 'add edges between events of this type', or the conjunction of the two definitions.

• What statistics are most meaningful to surface? e.g., min/mean/median/max, or a more nuanced approach like the intervals frequency distribution visualization

[Issue created by bestchai: 2016-12-02]

Dialog is displaying collapse button for nodes that are not collapseable

When collapsing nodes based on time differences, some single nodes are showing the collapse button on their dialog boxes. This is a wrong behavior because this node was not collapsed in the first place.

[Issue created by mhnnunes: 2016-06-22]
[Last updated on bitbucket: 2016-07-06]

[Comment created by mhnnunes: 2016-07-06]
Resolved in commit 8a9a97ae842c152c1ae4f71612d3017dd3fe99e5

The "compressed" edges do not change width when included in search result

At the moment, when the edges that have been compressed (shown in the attached image) and are included in a search result do not change width so it becomes hard to distinguish whether or not they are part of a search result.

There are two possible solutions to this:

1. We can add code which also supports changing the width of the paths in the search result.
2. Or, we can make all the edges and nodes not included in the search result to have a lower opacity than the search results, and darken the edges of the search results.

The solution could be a combination of both.

[Issue created by harjeetlalh: 2016-09-18]

Host dialogs are inconsistent when some processes are hidden

After creating a bunch of hidden hosts, when clicking on the remaining hosts, sometimes a dialog of the wrong colour and at the wrong position appears. Sometimes no dialog appears at all.

[Issue created by grahamstl: 2017-04-30]
[Last updated on bitbucket: 2017-05-06]

[Comment created by grahamstl: 2017-05-06]
Fixed in commit 859379c

Fixing this turned out simply to be a matter of reverting 1e2864e, where it was introduced. That is the commit that was the 'solution' to the problem that Matheus and I struggled with where hostname tooltips would appear diagonally away from the host nodes. It appears that after upgrading to d3 v4.8, that problem was no longer present, so it seems to have been a d3 bug all along! (Or d3 had an unintuitive API which has now been corrected.)

Underscore is not being drawn on search bar

If the user tries to type an underscore character in the search bar, this character will not be drawn (it will not be displayed). In it's place, a blank space is shown.

Bug existed in the original ShiViz. Might be because the underscore is drawn below the search bar border.

[Issue created by mhnnunes: 2016-06-08]

[Comment created by mhnnunes: 2016-06-08]
Fixed on commit 5580fc44bd0cd9e93fbc85e60cbcebc8b38653bc

Update documentation, especially for input log format and regular expressions

Reorganize documentation to point to a page on the wiki with more information about the log formats.
Both the landing page, and the (?) next to the log input button in tsviz should direct users to this wiki.

Use these wiki pages:
https://bitbucket.org/mhnnunes/tsviz/wiki/Home
https://bitbucket.org/mhnnunes/tsviz/wiki/LogFormat

[Issue created by grahamstl: 2017-04-30]
[Last updated on bitbucket: 2017-05-02]

[Comment created by grahamstl: 2017-05-02]
Resolved in commit 54213c8

The current version of the wiki is commit 53b769735188885500651da806b123338dd657c2

Much of the wiki has been copied over from ShiViz, but I made as many updates and corrections as I could find.

Event dialogs don't disappear when moved out of view

Click on an event node that opens a dialog. Then scroll horizontally (if there are so many processes that some are out of view), or vertically (if there are enough events). When the column for the selected event log goes off to the right, the nodes and edges disappear, but the dialog remains. When the dialog goes above the

Ideally, it would either hide or disappear when going out of the view area.

[Issue created by grahamstl: 2017-04-30]
[Last updated on bitbucket: 2017-05-02]

[Comment created by grahamstl: 2017-04-30]
Also, host dialogs don't disappear when scrolled horizontally out of view

Thread lines at top do not start at 0

All thread lines should start at ruler 0 mark. They currently look like this:

[Issue created by bestchai: 2016-08-04]
[Last updated on bitbucket: 2016-10-04]

[Comment created by ashayas: 2016-10-04]
Fixed issue #9 and #30

→ <>

Node info sidebar tab edge

The edge displayed between the two nodes on the "Node Information" tab on the sidebar is slightly off.

[Issue created by mhnnunes: 2016-11-20]
[Last updated on bitbucket: 2016-11-23]

[Comment created by ashayas: 2016-11-23]
Fix Issue #37

→ <<cset 50da7a9>>

Reject logs that violate local process timing constraints

We already reject logs that do not have monotonically-increasing-by-1 vector clock stamps for each process. But we should ensure these constraints:

1. Timestamps should increase monotonically for a single process.
2. Timestamps should increase monotonically between all happens-before events

E.g., given this regular expression:

(?<timestamp>(\d*)) (?<event>.*)\n(?<host>\w*) (?<clock>.*)

This log violates rule (1) and should be rejected (during parsing):

10001000 x1
a {"a": 1}
10002000 x2
a {"a": 2}
10001000 x3
a {"a": 3}

Instead, the second edge doubles back, overlapping the last node with the first.

Here's an example of the violation of constraint (2), which assumes the process's system clocks are sufficiently-synchronized for the visualization to be meaningful:

10002000 a sends message to b
a {"a": 1}
10001000 b receives message from a
b {"b": 1, "a": 1}

[Issue created by grahamstl: 2017-04-22]

Remove 'Multiple executions regular expression delimiter:'

The log entry page still has the 'Multiple executions regular expression delimiter:' label and text box from ShiViz. These need to be removed.

[Issue created by bestchai: 2017-04-28]
[Last updated on bitbucket: 2017-05-01]

[Comment created by grahamstl: 2017-05-01]
It turns out that this feature DOES work for tsviz, the we want to put these back now.

This removal is reverted in commit ec1e3d3

[Comment created by grahamstl: 2017-05-01]
Resolve in commit 230e2c2

Enter on search bar should search

Currently, the enter button is not bound to search bar. A user would want this to search.

[Issue created by ashayas: 2016-09-18]

[Comment created by harjeetlalh: 2016-09-18]
Already addressed in commit 7d0f823