softvis-research/jqa-dashboard

Dashboard for analyzing and visualizing software artifacts' data scanned with jQAssistant and stored in a Neo4j graph database.

Software Analysis and Visualization Dashboard

A web-based dashboard for software analysis and visualization using software artifacts' data scanned by jQAssistant. This screencast shows how to scan, analyze, and visualize software artifacts' data with the open source stack (jQAssistant, Neo4j, React, D3).

Demo

Interactively explore jUnit in this online demo of the dashboard.

Prerequisites

The dashboard requires a Neo4j database where the information of the software to be analyzed has been imported with jQAssistant. There are three possibilities how to achieve this.

1. You can use jQAssistant to extract the data of your software project as described here. After successful extraction, the data has to be enriched with the concepts in jqa-dashboard/data/jqassistant. Therefore, copy this folder in your project root and run $ mvn jqassistant:analyze. Then you can start the Neo4j database with $ mvn jqassistant:server. The dashboard uses the bolt driver to connect to the database (bolt://localhost, port 7687). You can change the default connection settings on the settings page of the dashboard.
2. Alternatively, you can use one of the pre-built Docker images containing a Neo4j server with sample data (spring-petclinic or jUnit) as described below.
3. Or you import the dumps provided in the data directory directly into your Neo4j database with $ neo4j-admin load --from=<path-to-neo4j-dump> [--force=true].

Installation

Clone the repository and execute the following command.

$ npm run install-dashboard

After installation you can run the dashboard by executing this command.

$ npm run dashboard

Update

For updating all packages and dependencies after running git pull you can execute this command.

$ npm run update-dashboard

Docker

Dashboard

The dashboard can also be run via Docker. Therefore, you can use these pre-built Docker images.

$ docker run --env PORT=3000 -it visualsoftwareanalytics/jqa-dashboard:dashboard

Data

If needed, you can use one of these commands to additionally run a Neo4j server with sample data already loaded.

$ docker run -it -p 7474:7474 -p 7687:7687 visualsoftwareanalytics/jqa-dashboard:neo4j-junit

or

$ docker run -it -p 7474:7474 -p 7687:7687 visualsoftwareanalytics/jqa-dashboard:neo4j-petclinic

Contributing

⇄ Pull requests and ★ Stars are always welcome. We kindly invite you to read the contributing guide to get started.

External Credits

• CoreUI
• Nivo
• React
• React Table
• Graph App Kit
• Neo4j

Publications

• David Baum, Pascal Kovacs, Richard Müller: Fostering Collaboration of Academia and Industry by Open Source Software, SE20 Software Engineering, 2020.
• Richard Müller, Dirk Mahler, Michael Hunger, Jens Nerche, Markus Harrer: Towards an Open Source Stack to Create a Unified Data Source for Software Analysis and Visualization, 6th IEEE Working Conference on Software Visualization, 2018.
• Tino Mewes: Konzeption und prototypische Implementierung eines web-basierten Dashboards zur Softwarevisualisierung, Masterarbeit, 2018.

You can find a complete list of publications on our website.

jqa-dashboard's Issues

Restrict database connection to bolt protocol

It should only be possible to use the bolt protocol (no http) to connect to the Neo4j database. At least the .env file and the AbstractDashboardComponent have to be changed.

Relative context path

We're preparing an integration of the dashboard as a plugin of jQA, a project has been created in the jQA contrib area (https://github.com/jqassistant-contrib/jqassistant-dashboard-plugin). Therefore the embedded Neo4j server in jQA has been extended such that plugins can provide static resources (HTML/JS/CSS etc.) which are then available in a specific context path. For the dashboard plugin the proposed URL after starting the server is http://localhost:7474/jqassistant/dashboard.

During evaluation it turned out that the current implementation of the dashboard relies on accessing resources etc. in the root context, e.g. http://localhost:3000. Is there a chance to change this behaviour, e.g. by specifyfing a configuration containing a relative root path?

Neo4jError in online demo

The online demo (http://139.18.211.212:3000/#/dashboard) does not work for me. The following error I get every time without clicking on anything.

×
Unhandled Rejection (Neo4jError): WebSocket connection failure. Due to security constraints in your web browser, the reason for the failure is not available to this Neo4j Driver. Please use your browsers development console to determine the root cause of the failure. Common reasons include the database being unavailable, using the wrong connection URL or temporary network problems. If you have enabled encryption, ensure your browser is configured to trust the certificate Neo4j is configured to use. WebSocket `readyState` is: 3
▼ 4 stack frames were expanded.
captureStacktrace
/node_modules/neo4j-driver/lib/browser/neo4j-web.js:22600
new Result
/node_modules/neo4j-driver/lib/browser/neo4j-web.js:22466
Session._run
/node_modules/neo4j-driver/lib/browser/neo4j-web.js:23000
Session.run
/node_modules/neo4j-driver/lib/browser/neo4j-web.js:22978
▲ 4 stack frames were expanded.
DashboardModel.readTestCoverageMetrics
/src/api/models/Dashboard.js:323
  320 | readTestCoverageMetrics(thisBackup) {
  321 |     var testCoverageMetrics = [];
  322 | 
> 323 |     neo4jSession
      | ^  324 |         .run(this.state.queryStringTestCoverage)
  325 |         .then(function(result) {
  326 |             result.records.forEach(function(record) {
View compiled
Dashboard.componentDidMount
/src/views/Dashboard/Dashboard.js:78
  75 | dashboardModel.readActivityMetrics(this);
  76 | dashboardModel.readStaticCodeAnalysisPMDMetrics(this);
  77 | dashboardModel.readHotspotMetrics(this);
> 78 | dashboardModel.readTestCoverageMetrics(this);
     | ^  79 | 
  80 | this.setState({
  81 |     queryStructure: localStorage.getItem(
View compiled
▼ 18 stack frames were expanded.
commitLifeCycles
/node_modules/react-dom/cjs/react-dom.development.js:18115
commitAllLifeCycles
/node_modules/react-dom/cjs/react-dom.development.js:19674
HTMLUnknownElement.callCallback
/node_modules/react-dom/cjs/react-dom.development.js:147
invokeGuardedCallbackDev
/node_modules/react-dom/cjs/react-dom.development.js:196
invokeGuardedCallback
/node_modules/react-dom/cjs/react-dom.development.js:250
commitRoot
/node_modules/react-dom/cjs/react-dom.development.js:19898
(anonymous function)
/node_modules/react-dom/cjs/react-dom.development.js:21446
unstable_runWithPriority
/node_modules/scheduler/cjs/scheduler.development.js:255
completeRoot
/node_modules/react-dom/cjs/react-dom.development.js:21445
performWorkOnRoot
/node_modules/react-dom/cjs/react-dom.development.js:21368
performWork
/node_modules/react-dom/cjs/react-dom.development.js:21273
performSyncWork
/node_modules/react-dom/cjs/react-dom.development.js:21247
requestWork
/node_modules/react-dom/cjs/react-dom.development.js:21102
scheduleWork
/node_modules/react-dom/cjs/react-dom.development.js:20915
enqueueSetState
/node_modules/react-dom/cjs/react-dom.development.js:11596
LoadableComponent.push../node_modules/react/cjs/react.development.js.Component.setState
/node_modules/react/cjs/react.development.js:336
update
/node_modules/react-loadable/lib/index.js:240
(anonymous function)
/node_modules/react-loadable/lib/index.js:250
▲ 18 stack frames were expanded.
This screen is visible only in development. It will not appear if the app crashes in production.
Open your browser’s developer console to further inspect this error.

Attached the console output as log file.
The problem occurs when using chromium. With firefox it seems to work without problem.
139.18.211.212-1582110922207.log

Activate production build in dockerfile of dashboard

At the moment the development build is used for the docker image of the dashboard.

Compiled successfully!
You can now view @coreui/coreui-free-react-admin-template in the browser.
Local: http://localhost:3000/
On Your Network: http://XXX.XXX.XXX.XXX:3000/
Note that the development build is not optimized.
To create a production build, use npm run build.

Please change it to production build.

Package explorer is only partly visible

The package explorer used in the structure and hotspots view is only partly visible. However, it should use the available space. If it is possible with a manageable effort, a horizontal and vertikal scrollbar should be added.

Externalize Neo4j settings into .env file

To provide a fully automatic demo of the dashboard via Rancher it is necessary to set the Neo4j settings (URL, username, password) externally in a .env file.

By default the dashboard uses the settings of the .env file. However, it should still be possible to customize them on the settings page. Therefore, a checkbox labeled "Customize database settings" is added to the settings page. If the checkbox is enabled the textboxes for the database settings become editable. Details on how to implement this are provided here: https://www.digitalocean.com/community/tutorials/how-to-deploy-a-node-js-and-mongodb-application-with-rancher-on-ubuntu-16-04 Finally, the Neo4j docker files have to be adapted accordingly.

Please support Neo4j Desktop API

So that the dashboard can be installed as a local or remote graph app (online app)

And connect to your local databases, other examples

https://yworks.com/neo4j-explorer

see: the API above this anchor

https://github.com/neo4j-apps/graph-app-starter#distribution

Idea: Visualization Wizard

Context

Currently, there are some predefined visualizations with predefined cypher queries possible. It would be great if there was a feature that guides you to your own, custom visualization using your own custom cypher query.

Idea

Implement a "Visualization Wizard" that helps you to create your own custom visualization by

1. Listing possible visualizations (form a set of predefined visualizations) in kind of a gallery
2. Listing and describing the required cypher query result that is needed for each visualization (plus: including an example query)
3. Offering an "Create" button that adds a new entry in the menu with the custom visualization as new web page

In the first step, I think it would be enough for a newly created, custom visualization if there is a new web page with the chosen visualization and an enabled expert mode (see #4).

Things to think about

Where should the customization data be stored? A possible way could be using HTML5 Web Storage (like , as I think, the "Favorites" section in the Neo4j Frontend does work to save and load custom cypher queries). Including a simple file export / file import function would be good enough for the start.

Idea: Expert mode

Context

Currently, the existing cypher queries for the visualizations are all hard-coded. It would be great if experts (people that know how cypher works) could manipulate those queries directly on the dashboard (on their own risk :-) ).

Idea

Introduce "expert mode" for each visualization section:

1. Add a switch that, when activated, shows a text field with the cypher query that is executed for the visualization
2. Add an execute button next to the text field that can fire the cypher query against the Neo4j database
3. Update the visualization with the results of the executed query

Beginners problem collection

I have reported several small issues in the Neo4j community forum: https://community.neo4j.com/t/how-to-get-started-with-jqa-dashboard/6568

Adjust order of error categories in radar chart of static code analysis view

Put error categories in radar chart in a descending order (clockwise) with regard to their number of violations.

Ensure successful building via CI

To ensure successful builds, you could add a Travis CI file named .travis.yml into the project's root directory and let the project build automatically after every commit via https://travis-ci.org/.

Maybe a build file like this could help (I don't know the exact node.js requirements):

language: node_js

node_js:
  - "iojs"
  - "7"

install:
  - "npm install"
  - "npm install graph-app-kit --registry https://neo.jfrog.io/neo/api/npm/npm"
  - "npm run build"

This method also integrates nicely with future pull requests where a PR can only be merged if the application builds (and tests run) successfully.

Neo4j settings should accept empty username/password and provide default URL

After initially starting up the dashboard the user is asked for connection details to the Neo4j server.

As the default situation will be a user running the Neo4j server which comes embedded with jQAssistant (mvn jqassistant:server or jqassistant server from the CLI) the settings should provide sensible defaults such that just "Save" can be clicked:

• URL: bolt://localhost
• Username:
• Password:

Furthermore there's a Name required. It should be evaluated if this is necessary and if so a default value (e.g. My Project can be specified).

Broken concepts link in README

@rmllr The concepts link in the README file (http://buschmais.github.io/jqassistant/doc/1.6.0/#_concepts) is broken. Can you please update the link? Thanks in advance!

Layout problems with Latest 20 commits

The "latest 20 commits "escapes" from its surrounding container if "Rows" (i.e. page size) is set to a value higher than 10:

A scrollbar should be allowed in this case.

Furthermore it might be a bit confusing for non-expert-mode users that the view is named "Latest 20 commits" but more than 20 rows can be set as page size.

Pin neo4j docker image versions in provided Dockerfiles

The jQA dashboard does not work with the latest version of Neo4j. The latest version of Neo4j, currently version 4.0.5, is used as the base image in the Dockerfiles provided. Therefore, it's necessary to pin the used Neo4j docker images in the provided Dockerfiles neo4j-junit and neo4j-petclinic as it's best practice (https://nickjanetakis.com/blog/docker-tip-18-please-pin-your-docker-image-versions).

It still has to be investigated, which is the last still working version but probably the latest version of Neo4j 3.5 should be fine. On my computer, version 3.5.3 is working well.

Knowledge distribution reports error

Knowledge distribution cannot be opened with latest dashboard version on master as the following error is reported:

To reproduce it see #25.

Broken layout for commits over time

The view for commits over time has a broken layout regarding the time scale:

For reproducing:

• Clone the jQA JUnit demo from https://github.com/jqassistant-demo/junit4/
• Checkout branch jqassistant/demo
• Run "mvn install"
• Open the dashboard: "Resource Management" -> "Activity"

The problem has been observed with

• Firefox 67.0 (64-bit)
• the dashboard in the online demo and built from current master branch

Architecture/Structure view should be available without Git

For this, the structure view needs to be decoupled from the hotspot query, e.g. by having a own simpler query.