tsdataclinic/trec

Transit Resilience for Essential Commuting (TREC)

trec's Issues

[UI] lists should only contain <li>

Adding an API server for map data

#30 will create a database of POIs/polygons. This issue will create an API layer for the UI to access those things as needed.

The API layer will serve two kinds of data:

• all data (not geographic data) each row as JSON data
• geographic data from each row as vector tiles

As part of this ticket, we can stick to FastAPI and use PostGIS functions directly for vector tile rendering: https://medium.com/nyc-planning-digital/using-the-new-mvt-function-in-postgis-75f8addc1d68

rough sketch of API endpoints needed:

• get a list of all cities
• for a given city:
  • get all hospitals
  • get all stops
  • get all routes
  • get flood layers

How will the API be used?

• on load, load all data for a given city
• send bounding box from UI to API and get back active city within bbox

Add Fathom analytics

Add City metadata table to db

Add a table to the database with

• City/MSA name
• City/MSA code
• Display name
• Centroid (point)

Route filter showing extra categories

Describe the bug
The Route filter for a city currently shows Rail and Ferry as options (for NYC and HR respectively) but those stops are not included in the stops view.

To Reproduce
Steps to reproduce the behavior:

1. NYC -> NYC lines -> Rail -> SI shows no stops on the map

Expected behavior
Rail and Ferry shouldn't be shown as options in the filter

Create second sidebar component

Need to create a new sidebar that sits to the right of the existing sidebar. Create a new component or edit the existing ContextPane component to do so.

Pipeline improvements

• Add checks in the pipeline to see if raw/processed data exists to avoid re-fetching existing data
• Add a force/clean parameter in the pipeline to bypass checks and run the entire process from scratch
• Modify parsing of city parameter in run_pipeline to accept multiple cities

The improvements are to make it easier to add multiple cities at once to the config and re-run the pipeline optimally without having to regenerate existing data.

Only show relevant filter toggles

Is your feature request related to a problem? Please describe.
As we include more cities, we need to be able to present filters (i.e. transit lines and flood layer) only relevant to that city.

Describe the solution you'd like
Currently we display all the toggles. But we would like to hide the toggles for other cities and only display the ones for the selected city.

For example: when NYC is selected up-top,

• 2050 Hampton Roads Floods (Projected) toggle should be hidden
• Hampton Roads lines dropdown component in Filter by Transit Line should be hidden

Writing Route list as an array to the DB

Route list associated with each stop is currently a string. Change it to an array and test how we can write it to the DB.

Full USA view on landing page

Finding flood layer data for additional cities

Create new about page

Convert About page to look like this one: covidinafrica.tsdataclinic.com/about

• add an /about route handler to the app here: https://github.com/tsdataclinic/TREC/blob/main/app/src/App.tsx
• add a new About component in this directory: https://github.com/tsdataclinic/TREC/tree/main/app/src/components
• Use the page sections from this page as a guide: https://covidinafrica.tsdataclinic.com/about and use the current About modal as the source of the text
• add a link to the text "TREC - Census TOP" linking back to "/"

Make sidebar elements city-specific

Right now, features on the sidebar always show up.

This change will make those features appear based on which city is being displayed.

Display Bay area Flood layer

Is your feature request related to a problem? Please describe.
San Francisco is the only city in our current list that does not have a hi-res Flood layer displayed.

Describe the solution you'd like
Display either the FEMA flood layer or relevant data from Cal-Adapt

add stop list to sidebar

Use MapLibre

Add loading state view to Route Summary view

When selecting a Route to view its summary in the RouteSidebar component, the loading state view is pretty ugly.

Problems:

• if there was no previous route summary shown, the loading state view changes width when the data is finally loaded
• if there was a previous route summary shown, the component displays that view before loading the new data

Acceptance:

• show a loading state for all the UI items: total stops and each of the horizontal bar charts
• for the horizontal bar charts, hide numbers instead of showing NaN
• eliminate the width-changing jank when data state goes from loading to loaded

Optional:

• add animated/visually informative progress indicators

Move data to a database

Adding incremental datasets currently requires a new file + a rebuild.

The way to change that for this project will be to add a spatial database (PostGIS) . Other approaches considered were going direct to vector tiles (but that wouldn't change the one file per city requirement). Spatialite and GeoPackage were considered but the planned complexity of this data will be better suited with a more traditional RDBMS.

This issue does not include work to create an API layer for data access. That will be conducted in a future issue.

This does not preclude further extensions to vector tileset formats, but those will be addressed separately.

---

Database setup

This issue will involve setting up a PostGIS database on an EC2 server. To start, we can pull the Docker image.

Proposed database architecture

There are several data files that TREC UI loads.

• POI: transit stops (geojson)
• POI: hospital locations (geojson)
• Polygon: projected flood layer (mbtiles)

To cover the POI cases, there should be two tables: transit_stops and hospitals. This will involve reading the current GeoJSON files and translating the properties object into DB columns.

To maintain parity, the transit_stop table should have at least the following columns (derived from here):

        "stop_name",
        "city",
        "route_type",
        "routes_serviced",
        "flood_risk_category", 
        "flood_risk_value", 
        "job_access_category", 
        "job_count",
        "worker_vulnerability_category", 
        "worker_vulnerability_score"
        "access_to_hospital_category",
        "distance_to_hospital",
        "geometry"

In addition, to make the table work across multiple cities there will also be:

• city
• country (ISO-3166-1? add subdivisions with ISO-3166-2?)
• geometry (a direct port of the GeoJSON's geometry obj)
• other columns that currently don't exist in the GeoJSON - ask @kaushik12 for details

For hospitals, there is a similar process:

• bringing over the current set of properties (FEATURE_NAME, FEATURE_CLASS)
• adding city, country, and geometry columns as with transit_stops table

For flood layers, we currently have polygons for NYC and Hampton Roads so we could go with a table with the same city and country keys and a geometry column which contains the polygons.

For other cities, where we might not have city specific data, we could fall back on using NOAA SLR scenario layer or Floodplain layer which are national datasets. To fit this into the same table structure, we could write out subsets of these as individual rows for each city and include this write operation in the data pipeline.

[UI] City selection dropdown should have associated label elements

Sourcing Datasets

Scripts to get the following datasets:

• FEMA National Risk Index
• CDC Social Vulnerability Index
• EJ Screen Point data (Schools, places of worship, hospitals)
• LODES data
• Shape files (Block grp, tract) of NYC MSA
• NYC DEP's stormwater flood data
• Transit data (Stops, routes)
• First St. foundation flood map
• Census tables

About page redesign

Add missing transit agencies in the Bay Area

Is your feature request related to a problem? Please describe.
Transit Land data for bay are doesn't include all the agencies here. Data from agencies south of Palo Alto isn't included.

Describe the solution you'd like
Include these additional sources in the pipeline and re-run.

Add images to about page

Census variables interpolation

For a given polygon(s), return the census variables by merging and interpolating the raw data

Inputs:

• Polygon shape(s)

Outputs:

• Data frame of aggregate census variables for these polygon(s)

Download data directly from app

• as GeoJSON
• as image?

Responsive layout

TREC is difficult to use on mobile, since the UI components hide the map.

A first pass to fixing this would be moving the sidebar and filter UIs off to the side until the user opts to see them.

Improve data transfer performance

Currently, stop-level data is transferred as uncached GeoJSON. This causes a big delay in performance.

The two possible solutions to try, are, in order:

• return data as vector tiles
• add a caching layer

Add distance to nearest hopital

As we expand the data being displayed on the front end, the distance to the nearest hospital from a stop could be useful. Add this calculation to the add_hospital_access in features/build_stop_features

RouteSummary for multiple route selection

Currently the route summary sidebar appears from clicking on a route in the tooltip. We'd want to make it possible to view summary based on route selection from the filter as well. The Filter though allows for multiple selections, so we'd have to figure out what to display when multiple routes are selected.

Allow switching climate risk tertiles between national and city-level

Currently the climate risk tertiles (High/Med/Low) are based on the national distribution of the risk scores. This is relevant in some circumstances such as understanding if the city has significant risk from a particular type of disaster, but perhaps in the context of prioritization within a city, one might be interested in understanding where the highest climate risks are within the city.

Tasks:

• Backend: Additional columns for climate risk to include city-level tertiles
• Frontend: An easy UI element to switch the respective column to use

Make filter slider work in all directions

Transit walkshed

A function that calculates the walk-shed around a station.

Inputs:

• Point(s): Starting with calculations around a single point. It's fairly easy to do a union of polygons if we choose to include multiple points(entries/exits/platforms) per station
• Road-network: 'walk' layer from Open Street Maps using osmnx library
• Time: in minutes (eg. 5, 10, 15, etc.)
• Walking Speed: Typical assumptions are around 1.5-3 mph (2.5-5 kmph). We can pick a fairly high value here to be more conservative.

Outputs:

• A (Convex) polygon that envelopes all the nodes reachable from the provided point in the given time.

Non-polygon transit walksheds

Due to disconnected components in the walk graph, the transit walksheds for some stops are not polygons creating issues down the line.

[UI] Modal on load

Add a dismissible modal on page load to describe TREC

Route summary

To show route level summaries, need to write additional calculation based on the geojson data passed into the app.

Create a hook that, for each route, create an object that contains:

• number of stations
• a count of each station based on variable (flood risk, access to hospital, access to jobs)
• list of stations

Searchable dropdown for many cities

For many cities, it might be easier to search rather than select from a picker