ragtagopen/marchon-map

Map for March On https://www.wearemarchon.org

March On map

Draw March On affilates and allied events on a Mapbox map, for embedding on March On.

Google Sheets to GeoJSON

Mapbox GL JS doesn't allow access to the entire feature set, only features in the current view (mapbox/mapbox-gl-js#2481). This map has a small number of features, and we need access to all of them to find the nearest to a user's location. Run an AWS Lambda script daily to get data from a Google sheet and save as GeoJSON on S3.

AWS Lambda function creates GeoJSON from a Google sheet:

• get data from Google Sheets via Google API
• get GeoJSON from public S3 URL
• geocode locations with Mapbox's Geocoder API
• write GeoJSON to S3

GeoJSON to map

map.js uses Vue.js and Mapbox GL JS to create the map.

• create Mapbox GL map with MarchOn's custom style
• load GeoJSON with affiliate data from S3 (updated daily by Lambda function)
• request user's location; if available zoom map to closest affiliate
• zoom to closest affiliate on results from Mapbox Geocoder control
• use Haversine formula to find closest affiliate
• show nearest/clicked/moused over affiliate info in card to the right of map

Deployment

Build deployment package

The image resizing code uses Pillow, which contains platform-specific C code. When deploying, make sure you include the Linux version in the zip file. The easiest way to do this is to create the deployment package on Linux; get a Docker container if you don't have access to a Linux box. Alternatively, you can pip install Pillow -t linux-pillow on Linux, then copy the resulting packages into the zip. You can do this just once, then freshen marchon.py as needed.

create directories for Linux deployment package

mkdir lambda-linux
mkdir lambda-linux/lambda
cp lambda/marchon.py lambda/requirements.txt lambda-linux/lambda

run Ubuntu docker container with python 3.6

docker run -v /path-to/marchon-map/lambda-linux:/lambda-linux -it --rm tomersha/docker-ubuntu-14.04-python-3.6.2

activate python 3.6, install zip, install packages, and create zip archive

pyenv shell 3.6.2
apt-get install zip
cd lambda-linux/lambda
pip install -r requirements.txt -t .
zip ../linux-lambda.zip -r .
exit

Deploy to AWS

Upload linux-lambda.zip to AWS function definitions, through AWS console or using the following CLI command

aws lambda update-function-code --function-name marchon-events-geojson --zip-file fileb://linux-lambda.zip
aws lambda update-function-code --function-name marchon-map-geojson    --zip-file fileb://linux-lambda.zip

Local setup for testing

Google sheet to GeoJSON (python)

Clone this repo and get python 3.6 set up.

Create a Google project and get API key

Set these in environment

GOOGLE_API_KEY
SHEET_ID
MAPBOX_ACCESS_TOKEN

From the lambda directory:

• install requirements: pip install -r requirements.txt -t .
• run python test_events.py > ../events.json

map (Vue.js)

Run a local web server: python -m SimpleHTTPServer

Go http://localhost:8000 in browser

To load a different GeoJSON data file, edit const geojson = pegasus(https://s3.amazonaws.com/ragtag-marchon/${mapjs.getAttribute('data-filename')}`);` in map.js

marchon-map's Issues

Create a map to display upcoming protests

Requirements:

• a map like the events map in events.html, but using the output of the lambda function described in #29
• Events should gray out the day after the event
• For the moment, use the star image as the map marker

Add try/except for case where "Link" cell is empty

Error handling if spreadsheet row lacks a Link (column 3)

index.html should use eventLink as well as motpLink

index.html only looks at motpLink for events; should also look at eventLink. Depends on fixing #53 first.

events.html: make border transparent

Make the pink border around the map (body color) transparent.

notify on geocoding failure

Sometimes MarchOn puts locations in the spreadsheets that the Mapbox Geocoder can't geocode (for example: Loudon County, TN). The Lambda function logs a warning, but it'd be nice to also notify MarchOn somehow so they can update the spreadsheet data.

The Lambda runs twice a day (10am/10pm PST), and ideally would send only one notification containing all geocoding errors. Email is probably the best way to get in touch with MarchOn (addresses TBD).

Once the new Lambda function is ready, set it up to run every hour

Once #29 is done, set a Lambda function to run every hour

new icon for houseparties

We need a new icon (probably smaller than 15x15) to indicate houseparty events.

• 15x15 or smaller
• svg format
• inline styles

See https://www.mapbox.com/help/studio-troubleshooting-svg/ for more info

Portland ME event showing up in Portland OR

This event

    {
      "properties": {
        "source": "events",
        "affiliate": false,
        "name": "WM First Anniversary Celebration",
        "eventDate": "1/21/2018",
        "eventLink": "https://www.facebook.com/events/918996544925020",
        "location": "Portland, ME",
        "host": "Women's March--ME",
        "contactName": "Genevieve Morgan",
        "contactEmail": "[email protected]",
        "facebook": "",
        "twitter": "",
        "instagram": "",
        "motpLink": "https://marchonthepolls.org/event/womens-march-on-washington-maine/"
      },
      "geometry": {
        "type": "Point",
        "coordinates": [
          -122.588947,
          45.485973
        ]
      },
      "type": "Feature"
    },

is showing up in Portland OR, but pretty clearly belongs in Portland ME

events.html: "Find out more" link missing

If the event has a link, the info box should say 'find out more" with the link. Looks like events.html wants a field motpLink, but the JSON file has eventLink

mailto links on the events map are built as relative href links

email icon links are not mailto links but rather full links pointing back to the github.io site. for example, the washington dc "reunion march- let them know we are still here" event has an envelope icon linking to

https://ragtagopen.github.io/marchon-map/[email protected]

Better handling of past events on index.html

index.html always shows links to associated events, even those in the far past. Figure out what the threshhold should be and then fix the code.

Gray out past events

Need a grayed-out version of the star, and display it if the event is in the past.

Event with no link or location showing up on the map

In the JSON, it appears to be the first item. I'm guessing we need to skip both the header rows in the Google sheet.

{
"id": "81f7e285696a20daf4649e70e19d5bd9",
"properties": {
"name": "Name",
"eventDate": "Date",
"eventLink": "Link",
"city": "City",
"state": "State",
"country": "Country",
"street_address": "Address",
"placeName": "Address 148&150, San Vicente, Nayarit, Mexico"
},
"geometry": {
"type": "Point",
"coordinates": [
-105.25766200000001,
20.746404
]
},
"type": "Feature"
},

index.html map missing data?

the index.html version of the map is displaying only the affiliate events, and not all of them...

display ActionNetwork events

• filter GeoJSON features with source = "actionnetwork" into a list
• exclude these events from the affiliate / other lists
• add actionnetwork events to map with their own icon (depends on #7)

see
https://github.com/RagtagOpen/marchon-map/blob/master/map.js#L61 for how events are currently filtered into affiliate/not affiliate and https://github.com/RagtagOpen/marchon-map/blob/master/map.js#L83 for how these are added to map

get event data from ActionNetwork

API docs: https://actionnetwork.org/docs

with Python 3.6
will need at least a zip code and URL to the ActionNetwork page
what else is available?

This may require an API key from MarchOn.

update marchon.py to

[ ] get data from ActionNetwork
[ ] add feature to GeoJSON with source = actionnetwork

use the same property names as spreadsheet events (https://github.com/RagtagOpen/marchon-map/blob/master/lambda/marchon.py#L60)

Make Ragtag link open in a new tab

The Ragtag link in the map itself opens the ragtag.org page within the iframe. Please make it open either in another tab/page, or just not inside the iframe.

Exception handling if location fields are blank

Add error handling if location cells are empty (both in geocoding and in the default event name if no name is supplied)

Change color of house icon as per color map

Comment from March On:

Hi,
Just one note -- the aqua green might be the old color. It should be a light blue --

I'm sharing the current color sheet so it can match.

Otherwise, it's amazing!!
Thanks,
Beth
MOTP_colors.pdf

events.html: only one star type

This map shouldn't differentiate between affiliate and non-affiliate marches, leading to two changes:

• Use the pink star everywhere
• Remove the map legend

NB if this requires a change in map.js, please check that it only affects events.html (and not the map displayed by index.html at https://ragtagopen.github.io/marchon-map/index.html)

Vancouver, BC, CANADA fails to geocode

Discovered by @marionnewlevant and discussed in Slack.

Root cause is Vancouver, BC, CANADA changed to VANCOUVER, BC, CaliforniaNADA before geocoding, so Mapbox couldn't find it.

add filter by source to map

Add some sort of widget to filter by event type (source = "events" vs source = "actionnetwork")

• show filter widget only on events map, not affiliate map
• on filter, show events only from selected source

Make a Lambda function to handle the spreadsheet of protest events

To implement the new map of immigration-related protests, we need a new Lambda function that geocodes each row. The spreadsheet is at https://docs.google.com/spreadsheets/d/1EeObYkea1vGmAjCzm5vvmxCae89dDKBjDeUvZC24gK8/edit#gid=0

It should be straightforward to reuse the existing code for events, modified to deal with the new columns (City/State/Country rather than a single Location column)

Vue error and displaying wrong data

Poking around Oregon, I notice that the Springfield/Eugene star is showing what looks like the Coos Bay popup, and giving errors inthe console:

vue.js:584 [Vue warn]: Error in render: "TypeError: Cannot read property 'month' of undefined"

(found in )
warn @ vue.js:584
logError @ vue.js:1715
globalHandleError @ vue.js:1710
handleError @ vue.js:1699
Vue._render @ vue.js:4467
updateComponent @ vue.js:2765
get @ vue.js:3113
run @ vue.js:3190
flushSchedulerQueue @ vue.js:2954
(anonymous) @ vue.js:1815
flushCallbacks @ vue.js:1736
Async Call
microTimerFunc @ vue.js:1784
nextTick @ vue.js:1828
queueWatcher @ vue.js:3041
update @ vue.js:3180
notify @ vue.js:690
reactiveSetter @ vue.js:1015
proxySetter @ vue.js:3271
showFeature @ map.js:280
boundFn @ vue.js:196
(anonymous) @ map.js:186
e @ lodash.js:10350
a @ lodash.js:10397
f @ lodash.js:10385
setTimeout (async)
Cs @ lodash.js:6663
s @ lodash.js:10434
s @ map.js:632
Evented.fire @ evented.js:94
h @ bind_handlers.js:144
u @ bind_handlers.js:80

vue.js:1719 TypeError: Cannot read property 'month' of undefined
at Proxy.eval (eval at createFunction (vue.js:10518), :2:503)
at Vue$3.Vue._render (vue.js:4465)
at Vue$3.updateComponent (vue.js:2765)
at Watcher.get (vue.js:3113)
at Watcher.run (vue.js:3190)
at flushSchedulerQueue (vue.js:2954)
at Array. (vue.js:1815)
at flushCallbacks (vue.js:1736)
at

motpLinks unavailable for Action Network events

motpLinks should be copied to the JSON file for Action Network events.

Once this is done, you can remove the change in events.html that falls back to the eventLink.