integratedtransportplanning/gcvt

Greener Connectivity Visualisation Tool

Greener Connectivity Visualisation Tool

🚧 👷‍♀️ Work in progress 👷 🚧

The goal of the GCVT is to visualise and compare transport model outputs under different scenarios.

Installation

This project primarily uses two programming languages: Julia and JavaScript.

Some vestiges of R code remain and may be required for preparing your data for the visualisation tool (or you could prepare your data in a compatible way with some other language).

To install material other than R:

# install julia v1.x
# install yarn
# install caddy (or some other webserver)

# install Julia and JS dependencies
make setup

# move your scenario pack data to src/backend/data

# make tiles if you haven't yet done so
# tippecanoe and mbutil are what I use and a convenience script is provided in
# src/data-preparation/tiles.sh

# Run the development server:
make front &
make back &

# Access the map at http://localhost:2016

Docker - recommended for deployment:

# Ensure you have yarn and docker-compose installed and on your path
# Ensure processed data is in correct directory (e.g. ./src/backed/update_data.sh)

# Build frontend (see issue #80)
cd src/frontend/
yarn install
yarn run build

# Launch all services (will take a while)
cd ..
sudo docker-compose up --build

# Migrating WordPress installation is slightly tricky - see src/notes.md

gcvt's Issues

Centroid to centroid lines in OD viewer

Excessive output in default popups in network mode

To the point where it's overwhelming, reducing user-friendliness and moves the map view when you click on a link. Example view:

Zones don't use comparison palette

Legend visibility doesn't follow layer visibility

Story/setting recorder

Have a record button somewhere and a download button.

Download button provides a json or yaml file (whatever our metadata files look like).

observeEvent(input$record, {
  currentSettings = list(
    scenario = input$scenario,
    colourBy = input$colourBy,
    ...
  )
  recordings <<- c(recordings, currentSettings)
})

output$downloadRecordings <- downloadHandler(...)

Example: https://shiny.rstudio.com/gallery/file-download.html

Set initial zoom level (based on input data)

Scenario picker needs to move out and above of Network/Zone menus

Get properties of link in popup

assign markjd84

Design guided/simple story UI

Perhaps the user picks a scenario and a few one line stories are suggested somewhere?

"See difference in travel time now that border control is relaxed" -> Preset options (compare scenario with baseline in OD view with variable travel time)

Set initial zoom and center of map

Currently:

Allow user to colour all links the same colour

Comparison palette rounding sometimes causes boundary values to fall outside the palette

Proper solution is to separate presentation of bin sizes from actual bin sizes, but that's tricky with the existing leaflet functions.

Filter links by mode

Presentation

Software:

• Combine tools
• OD comparison incl. another scenario
• Scale to scenario wide var
• OD from/to selector

Presentation:

• Pick a couple of user questions and fake up some scenarios. (BCP, rail electrification)
• Prepare for followups: e.g. BCP -> rail gauge upgrades.

Extras for real presentation:

• Make tool web-accessible
• Screenshots
• Real data

Legend isn't as nice as the Leaflet one

The legend that R Leaflet gives you isn't provided by leaflet.js, it's something that the rstudio developers wrote.

We can fairly easily port it for mapbox because it doesn't depend on leaflet very much.

The function is here: https://github.com/cmcaine/leaflet/blob/012bec5a774547c6589beacfdc9a02d8d64fc9bb/javascript/src/methods.js#L765

I think the code to do it should live in the JS rather than R, then we can maintain more of a split between presentation code and the logic about what should be displayed.

Reduce initial load time

Reduce data volume:

• Compress data (better?) before sending it
• Half link data by only sending one direction of each link pair

Vector tiles:

• Create and serve tiles in the mapbox vector tile format (Only download detailed information for areas you zoom into)
• t_rex serves and generates MVT protobufs, but it's extremely slow at generating them, this could be because I'm generating from geojson, I think if I load the geom into postgis it will get simplified more appropriately and work faster.
• geojson_vt is very fast, but the total size of the tiles generated is huge (~30x bigger than original geojson)

Scale styles by the range of variable across all scenarios

Pre-compute this, probably.

Can just rbind all the scenarios together into one big dataframe and find the range for each column. Or a for loop, whatever.

reStyling functions need to be adapted to take the range. Probably take a names list of VARIABLENAME -> RANGETUPLE

e.g.

"SPEED" -> 15, 150

in R:

list(SPEED = c(15, 150))

Improve line styling

Don't want to fade to white. Try a larger range of widths.

Improve CSS for side menu

Add new origins/destinations - update the input zone file

Logos of funders

Generally improve OD viewer

Worklist for Wednesday meeting

• Working scenario picker
  • named list of scenario DFs
  • just a selection box for now
• Filter by mode
• Weight style NULL
• Highlight selected OD zone
• Mouseover centroid lines
• Bonus: graphs or scenario comparison

Make install use correct version of leaflet

Improve OD comparisons (centroid lines, palette)

It's a core user question.

Links are showing in blue prior to layer being turned on

Something to do with hideLayer() not quite getting called at right time?

Expected behaviour for `scale_to_range`

Not sure if this function is working as expected. More likely I suspect I've misunderstood some subtlety ;)

> test_range = 1:10
> scale_to_range(test_range, c(1,100))
 [1]  1.0 10.9 20.8 30.7 40.6 50.5 60.4 70.3 80.2 90.1
> test_range = 1:100
> scale_to_range(test_range, c(1,10))
  [1] 1.00 1.09 1.18 1.27 1.36 1.45 1.54 1.63 1.72 1.81 1.90 1.99 2.08 2.17 2.26 2.35 2.44 2.53 2.62 2.71 2.80 2.89 2.98 3.07 3.16 3.25 3.34 3.43 3.52 3.61 3.70 3.79
 [33] 3.88 3.97 4.06 4.15 4.24 4.33 4.42 4.51 4.60 4.69 4.78 4.87 4.96 5.05 5.14 5.23 5.32 5.41 5.50 5.59 5.68 5.77 5.86 5.95 6.04 6.13 6.22 6.31 6.40 6.49 6.58 6.67
 [65] 6.76 6.85 6.94 7.03 7.12 7.21 7.30 7.39 7.48 7.57 7.66 7.75 7.84 7.93 8.02 8.11 8.20 8.29 8.38 8.47 8.56 8.65 8.74 8.83 8.92 9.01 9.10 9.19 9.28 9.37 9.46 9.55
 [97] 9.64 9.73 9.82 9.91
> test_range = 20:30
> scale_to_range(test_range, c(1,10))
 [1] 1.0 1.3 1.6 1.9 2.2 2.5 2.8 3.1 3.4 3.7 4.0

It's that last one that looks odd to me?

Improve zones

• Remove slivers
• Make them contiguous
• Consistent level of simplification

I've already tried and failed on this so not promising anything - just thought I'd flag it as something I can look at. Any others I should look at?

Airports

Nice visualisation approach suggested by @cmcaine

I wonder if it would make sense to generalise very slightly, so that we could eg use for sea ports as well? They would have to be separable so the user doesn't confuse boats with planes, but could still use our line drawing code. Just a thought.

Error message when selecting Network links

This was the message I got on fresh install:

 Error in setStyleFast: could not find function "setStyleFast"
  53: styleByData [../app_common.R#144]
...

Just tried with the zones and seem to get the same issue.

OD centroid lines should be top values across all selected

(not N * top 20 for each zone)

Year slider

Does not currently do anything

Also see #34 - scenario picker outside 'layer control' groups

Both issues require some careful thinking about the UI. Might we (eg) want to see comparison of link datasets whilst viewing zone stats (or vice versa?). Is there a situation in which we'd want to see different years at same time? (can't think of one, but don't want to rule out)

Create metadata scheme

For ranges (define global, scenario, type, etc)

For colours (e.g. different colours for pollution)

For normalisation by?

Outliers in OD data skew palette + zones outside of EaP have bad data anyway

Africa and South China etc. skew the palettes.

Can exclude far away zones, exclude outliers from palette calculations (use a placeholder colour for them), clamp values, use quantiles, etc.

Visualise point data

Ability to visualise generic point data (could be a city, port or bus stop). The size and colour of the point to be reflect by data attributes. E.g. a city with a large population could be a larger dot.

Style link width by variable

Bin colours for continuous variables

Links should be offset

Try this: https://github.com/bbecquet/Leaflet.PolylineOffset

Comparison specific colour scheme

Two colour scheme with discontinuity at 0.

E.g. more red for -ve, more green for +ve. Would be better to find a colour blind friendly palette that still generally indicates good vs bad.

Or a three colour scheme with a neutral colour for the middle.

Shiny segfaulting on multiple page loads

Probably just need a version change for R

Listening on http://0.0.0.0:4400

 *** caught segfault ***
address (nil), cause 'memory not mapped'

Traceback:
 1: execCallbacks(timeoutSecs)
 2: run_now(timeoutMs/1000)
 3: service(timeout)
 4: serviceApp()
 5: ..stacktracefloor..(serviceApp())
 6: withCallingHandlers(expr, error = doCaptureStack)
 7: globals$domain$wrapSync(expr)
 8: promises::with_promise_domain(createStackTracePromiseDomain(),     expr)
 9: captureStackTraces({    while (!.globals$stopped) {        ..stacktracefloor..(serviceApp())        Sys.sleep(0.001)    }})
10: ..stacktraceoff..(captureStackTraces({    while (!.globals$stopped) {        ..stacktracefloor..(serviceApp())        Sys.sleep(0.001)    }}))
11: shiny::runApp("R/combined/", host = "0.0.0.0", port = 4400)
An irrecoverable exception occurred. R is aborting now ...

Change "TRUE" to "Same" in comparison legend

I already have code for this.

Need UI for picking dataset/scenario

type to filter style select box would do fine

Remaining mapbox integration tasks

Made some tweaks and got a test version running at http://integac7.miniserver.com:4402/

Performance seems to be good (so it's just my VM being sluggish!)

However, still need to

• test zone click+select interactivity, and 'scenario difference' modes
• draw legend
• work out how to show selected zones (fill-outline-color seems to be all that is supported, other options?)
• draw centroid lines
• render popup on links
• make mapbox the master branch
• render hover label on links and zones
• work out (and fix!) what is going on with Kaliningrad.

V_truck_ton is 0 for inappropriate links and skews width calculation

Set V_truck_ton, etc. to NA when appropriate? Calculate width differently?

Move common code for apps to shared file

From/to/sum selector on OD view

Display the rows, columns or sum of both.

Combine three Shiny apps to one, with navigation between

Also sort out hosting etc.

gcvt package cannot be installed

When matrix values are all 0, all lines are drawn (not just 20)

Provide scenario summary box

total GHG
total Local Air Emissions
total energy use by fuel type
total veh km
total travel cost / km
total travel time
total volume carried
total volume carried by mode