sharedstreets/sharedstreets-python

SharedStreets (Python)

SharedStreets Builder

The SharedStreets Builder application converts OpenStreetMap data to SharedStreets protocol buffer tiles.

SharedStreets uses this tool to generate and maintain a complete global OSM-dervied tile set. Users can operate the tool directly on their OSM or use a pregenerated global tileset provided by SharedStreets.

Support for non-OSM data sources has been moved to the sharedstreets-conflator tool.

Example use

java -jar ./sharedstreets-builder-0.1-preview.jar --input data/[osm_input_file].pbf --output ./[tile_output_directory]

Notes

The builder application is built on Apache Flink. If memory requirements exceed available space, Flink uses a disk-based cache for processing. Processing large OSM data sets may require several hundred gigabytes of free disk space.

Roadmap

• v0.1: OSM support
• v0.2: Add OSM metadata support for support ways per #9
• v0.3: add heirarchical filterfing for roadClass per sharedstreets-ref-system/#20

sharedstreets-python's Issues

GeoJSON ID length default

@migurski id trimming for GeoJSON defaults to 12 (of 32 total) characters. How do you feel about defaulting to the full-length id, and letting the caller specify a lower number? Not trimming by default seems like a less surprising behavior.

sharedstreets-python/sharedstreets/tile.py

Line 212 in 75205e2

def make_geojson(tile, id_length=12):

python example results in empty geojson file

@migurski just discovered that the GeoJSON CLI example in the docs returns a zero size feature collection:

sharedstreets-get-tile 16 10509 25324 > 16-10509-25324.geojson

I'm debugging, but also curious if there are any changes on your end that are pushed. We're bringing on a new person on end (hi @vipassana!) who will be working with the Python lib. Just want to make sure we have the latest and greatest!

Simplify purpose of sharedstreets-python

Simplify purpose of sharedstreets-python

Right now, the Python library not heading in a focused direction. We've already got a lot of tools built on the Javascript (NodeJS) side which allows server & browser compatibility.

The original purpose of the Python library was for simple to use SharedStreet methods:

• Hash a Message
• Convert a Geometry to ID
• Convert an Intersection to ID
• etc...

All other tooling should be done in NodeJS that way we have the code ready for the browser when we need it.

From my point of view, only the following core SharedStreets tools should exist in NodeJS:

• Convert PBF to JSON - sharedstreets-pbf (already completed)
• Convert JSON to GeoJSON - sharedstreets-geojson (port from python)
• Download tile from SharedStreets tile server - sharedstreets-tile (port from python)

SSL Error

Hello! I am getting the following SSLError:

SSLError: HTTPSConnectionPool(host='tiles.sharedstreets.io', port=443): Max retries exceeded with url: /osm/planet-180312/12-656-1582.geometry.6.pbf (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1091)')))

I am not sure if this is still maintained, but if it is, please I would like advice on how to proceed. I am currently using python 3.7.16 and I can't find a way to fix this after checking every version of all packages locally (or attempting to run the simple commands in the tutorial on Google collab).

Geocoding/Snapping points to sharedstreets ref

Hey! Awesome initiative, thanks for doing that.

I am a little bit lost though - I want to "snap" my POI to shared streets in order to get unique street segment id. Is this something that I need to do using sharedstreet tile myself? Do you think it makes sense to add that function, or at least example on how to do that, to shared streets?

sharedstreets.tile.get_tile doesn't get dedicated infraestructure for buses like BRT in Buenos Aires

I'am running Python 3.6.5
Using the python library on Buenos Aires City with sharedstreets.tile.get_tile(12, 1383, 2468) I get

There you can see Metrobus Network (Buenos Aires BRT) is missing in the geometries.

While in you online map you can see it loaded to sharedatreets network
https://platform.sharedstreets.io/-34.62128/-58.37974/16.467/

The OSM tag on the Metrobus BRT shows access=no, bus=yes as specified in the OSM docs

Use `.hexdigest()` rather than `.digest('hex')`?

Just digging around in here and testing things out. Looks like you have a node function here rather than the python function.

In index.py

def generateHash(message):
    return hashlib.md5(message).hexdigest()

See hashlib documentation.

Unexpected keyword argument "file"

Edit:
See this comment (#21 (comment)), the issue is that protobuf was upgraded on Mar 26 from 3.7.0 to 3.7.1. This upgrade breaks SS throwing the below error/traceback.

Original content:

---

In a fresh container I pip install ss:

pip install sharedstreets

It install successfully. I then run the following command:

sharedstreets-get-tile 16 10509 25324 > 16-10509-25324.geojson

The following error is emitted:

Traceback (most recent call last):
  File "/usr/local/bin/sharedstreets-get-tile", line 7, in <module>
    from sharedstreets.tile import main
  File "/usr/local/lib/python3.6/site-packages/sharedstreets/tile.py", line 4, in <module>
    from . import sharedstreets_pb2
  File "/usr/local/lib/python3.6/site-packages/sharedstreets/sharedstreets_pb2.py", line 148, in <module>
    options=None, file=DESCRIPTOR),
TypeError: __init__() got an unexpected keyword argument 'file'

This is the version of ss being used:

pip list | grep sharedstreets
sharedstreets      0.4.0

kickstarting python lib

@migurski just moving this thread over from sharedstreets/sharedstreets-ref-system#16 From the ticket:

So the SharedStreets tiles use length delimited encoding (the technique $GOOG recommends for files over ~1mb to allow stream processing of data). Turns out the Python Protobuf library doesn't have helper functions to parse these messages, but I found this example on how do to it:

https://www.datadoghq.com/blog/engineering/protobuf-parsing-in-python/

Just posted a quick example here using your compiled version of the .proto file:

https://github.com/sharedstreets/sharedstreets-python/blob/master/test.py

Let's use your example code as starting point for building tile loading functionality in the python lib.

Also @DenisCarriere want to create a punch list based on the core functionality from the JS lib so we can try and keep parity in the python version?