Giter Site home page Giter Site logo

udst / osmnet Goto Github PK

View Code? Open in Web Editor NEW
55.0 17.0 17.0 6.83 MB

Tools for the extraction of OpenStreetMap street network data

Home Page: https://udst.github.io/osmnet

License: GNU Affero General Public License v3.0

Python 100.00%
openstreetmap graph overpass-api street-networks network

osmnet's Introduction

OSMnet

Build Status Coverage Status Appveyor Build Status

Tools for the extraction of OpenStreetMap (OSM) street network data. Intended to be used in tandem with Pandana and UrbanAccess libraries to extract street network nodes and edges.

Overview

OSMnet offers tools to download street network data from OpenStreetMap and extract a graph network comprised of nodes and edges to be used in Pandana street network accessibility calculations and or UrbanAccess to connect GTFS transit networks to road networks.

Let us know what you are working on or if you think you have a great use case by tweeting us at @urbansim.

Library Status

Forthcoming improvements:

  • Tutorial/demo

Reporting bugs

Please report any bugs you encounter via GitHub issues.

Contributing to OSMnet

If you have improvements or new features you would like to see in OSMnet:

  1. Open a feature request via GitHub issues.
  2. Contribute your code from a fork or branch by using a Pull Request and request a review so it can be considered as an addition to the codebase.

Installation

conda

OSMnet is available on conda and can be installed with:

conda install osmnet --channel conda-forge

It is recommended to install via conda and the conda-forge channel especially if you find you are having issues installing some of the spatial dependencies.

pip

OSMnet can be installed via PyPI:

pip install osmnet

Development Installation

To install OSMnet from source code, follow these steps:

  1. Git clone the OSMnet repo
  2. in the cloned directory run: python setup.py develop

To update to the latest version:

Use git pull inside the cloned repository

Documentation

Documentation for OSMnet can be found here.

osmnet's People

Contributors

federicofernandez avatar knaaptime avatar pksohn avatar sablanchard avatar smmaurer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

osmnet's Issues

Tags-to-keep from Overpass API might need to be updated

Just a heads up that the tags that are identified as needing to be kept on this line:
https://github.com/UDST/osmnet/blob/master/osmnet/config.py#L61

are different than what is currently on the master branch of OSMnx. See the line in that lib here:
https://github.com/gboeing/osmnx/blob/8f9308fa43758baefe11f42d0388d22e9f4e2589/osmnx/globals.py#L29

Tags such as "area" may be useful in curating out polygons from ways results (which could lead to errors downstream when using this for routing purposes).

'vertices' must be a 2D list or array with shape Nx2 happening only for Banglaore?

Description of the bug

I have done the same over different regions of India and USA, I didn't get this issue untill trying over Banglore... Any thoughts and answers are most welcomed, Thanks!

OSM data (optional)

bbox = [12.8881, 77.5079, 12.9918, 77.6562]

Environment

  • Operating system: Windows 10 pro

  • Python version: 3.7.6

  • OSMnet version: 0.1.5

  • OSMnet required packages versions (optional): pandana 0.4.4, pandas 0.25.3, geopandas 0.6.3

Paste the code that reproduces the issue here:

import pandas as pd, geopandas as gpd
df = pd.DataFrame(
    {'id':[1522786113, 2227865111, 309601717, 1513928857, 2220792136, 6455354942],
        'Name': ['Neil', 'Nitin', 'Mukesh','Alpha','Beta','Office'],
     'Area': ['Valsad', 'Silvasa', 'Daman','Rajkot','Dui','Vapi'],
     'lat': [12.956550, 12.950360, 12.912047,12.955546,12.939653,12.928109],
     'lon': [77.540640, 77.581135, 77.586969,77.529658,77.542523,77.639337]})
pois = gpd.GeoDataFrame(df, geometry=gpd.points_from_xy(df.lon, df.lat))

network = osm.network_from_bbox(bbox[0], bbox[1], bbox[2], bbox[3],network_type='drive')
network=pandana.Network(network[0]["x"],network[0]["y"], network[1]["from"], network[1]["to"],network[1][["distance"]])
lcn = network.low_connectivity_nodes(impedance=1000, count=10, imp_name='distance')

network.precompute(distance + 1)
network.init_pois(num_categories=1, max_dist=distance, max_pois=7)
network.set_pois(category='my_amenity', x_col=pois['lon'], y_col=pois['lat'])
access = network.nearest_pois(distance=distance, category='my_amenity', num_pois=7)

bbox_aspect_ratio = (bbox[2] - bbox[0]) / (bbox[3] - bbox[1])
fig_kwargs = {'facecolor':'w', 
              'figsize':(15, 15 * bbox_aspect_ratio)}

# keyword arguments to pass for scatter plots
plot_kwargs = {'s':5, 
               'alpha':0.9, 
               'cmap':'viridis_r', 
               'edgecolor':'none'}

n = 1
bmap, fig, ax = network.plot(access[n], bbox=bbox, plot_kwargs=plot_kwargs, fig_kwargs=fig_kwargs)
ax.set_facecolor('k')
ax.set_title('Walking distance (m) to nearest boi around Bangalore-India', fontsize=15)
fig.savefig('images/accessibility.png', dpi=200, bbox_inches='tight')
plt.show()

Paste the error message (if applicable):

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-22-d557c4f5993e> in <module>
      1 # plot the distance to the nth nearest amenity
      2 n = 1
----> 3 bmap, fig, ax = network.plot(access[n], bbox=bbox, plot_kwargs=plot_kwargs, fig_kwargs=fig_kwargs)
      4 ax.set_facecolor('k')
      5 ax.set_title('Walking distance (m) to nearest {} around Berkeley/Oakland'.format(amenity), fontsize=15)

~\miniconda3\envs\ox\lib\site-packages\pandana\network.py in plot(self, data, bbox, plot_type, fig_kwargs, bmap_kwargs, plot_kwargs, cbar_kwargs)
    457         bmap = Basemap(
    458             bbox[1], bbox[0], bbox[3], bbox[2], ax=ax, **bmap_kwargs)
--> 459         bmap.drawcoastlines()
    460         bmap.drawmapboundary()
    461 

~\miniconda3\envs\ox\lib\site-packages\mpl_toolkits\basemap\__init__.py in drawcoastlines(self, linewidth, linestyle, color, antialiased, ax, zorder)
   1849         # get current axes instance (if none specified).
   1850         ax = ax or self._check_ax()
-> 1851         coastlines = LineCollection(self.coastsegs,antialiaseds=(antialiased,))
   1852         coastlines.set_color(color)
   1853         coastlines.set_linestyle(linestyle)

~\miniconda3\envs\ox\lib\site-packages\matplotlib\collections.py in __init__(self, segments, linewidths, colors, antialiaseds, linestyles, offsets, transOffset, norm, cmap, pickradius, zorder, facecolors, **kwargs)
   1357             **kwargs)
   1358 
-> 1359         self.set_segments(segments)
   1360 
   1361     def set_segments(self, segments):

~\miniconda3\envs\ox\lib\site-packages\matplotlib\collections.py in set_segments(self, segments)
   1372             _segments = self._add_offsets(_segments)
   1373 
-> 1374         self._paths = [mpath.Path(_seg) for _seg in _segments]
   1375         self.stale = True
   1376 

~\miniconda3\envs\ox\lib\site-packages\matplotlib\collections.py in <listcomp>(.0)
   1372             _segments = self._add_offsets(_segments)
   1373 
-> 1374         self._paths = [mpath.Path(_seg) for _seg in _segments]
   1375         self.stale = True
   1376 

~\miniconda3\envs\ox\lib\site-packages\matplotlib\path.py in __init__(self, vertices, codes, _interpolation_steps, closed, readonly)
    128         if vertices.ndim != 2 or vertices.shape[1] != 2:
    129             raise ValueError(
--> 130                 "'vertices' must be a 2D list or array with shape Nx2")
    131 
    132         if codes is not None:

ValueError: 'vertices' must be a 2D list or array with shape Nx2

Including OSMNet in other packages' requirements - now fixed

Problem

People sometimes get errors using setuptools to install packages that list OSMNet as a requirement.

For example, OSMNet is a requirement in Pandana's setup.py file. If you install Pandana on OS X or Linux using python setup.py develop, and OSMNet is not yet installed, you get the error below.

Solution

I poked around, and it seems like the problem is that the initial 0.1.0 release of OSMNet on pypi.org included files that no longer conform to the standard naming spec. This causes an error when setuptools tries to find an appropriate version to install.

There are several newer releases, none of which break the API in any way, so I went ahead and removed 0.1.0 from pypi. This immediately fixed the problem on my machine.

Let me know if you see any more issues related to this.

Original error message

Searching for osmnet>=0.1.2
Reading https://pypi.org/simple/osmnet/
Traceback (most recent call last):
  File "setup.py", line 133, in <module>
    'License :: OSI Approved :: GNU Affero General Public License v3'
  File "/anaconda3/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/anaconda3/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/anaconda3/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/anaconda3/lib/python3.6/site-packages/setuptools/command/develop.py", line 38, in run
    self.install_for_development()
  File "/anaconda3/lib/python3.6/site-packages/setuptools/command/develop.py", line 156, in install_for_development
    self.process_distribution(None, self.dist, not self.no_deps)
  File "/anaconda3/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 752, in process_distribution
    [requirement], self.local_index, self.easy_install
  File "/anaconda3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 782, in resolve
    replace_conflicting=replace_conflicting
  File "/anaconda3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1065, in best_match
    return self.obtain(req, installer)
  File "/anaconda3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1077, in obtain
    return installer(requirement)
  File "/anaconda3/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 667, in easy_install
    not self.always_copy, self.local_index
  File "/anaconda3/lib/python3.6/site-packages/setuptools/package_index.py", line 658, in fetch_distribution
    self.find_packages(requirement)
  File "/anaconda3/lib/python3.6/site-packages/setuptools/package_index.py", line 491, in find_packages
    self.scan_url(self.index_url + requirement.unsafe_name + '/')
  File "/anaconda3/lib/python3.6/site-packages/setuptools/package_index.py", line 831, in scan_url
    self.process_url(url, True)
  File "/anaconda3/lib/python3.6/site-packages/setuptools/package_index.py", line 373, in process_url
    self.process_url(link)
  File "/anaconda3/lib/python3.6/site-packages/setuptools/package_index.py", line 335, in process_url
    dists = list(distros_for_url(url))
  File "/anaconda3/lib/python3.6/site-packages/setuptools/package_index.py", line 99, in distros_for_url
    for dist in distros_for_location(url, base, metadata):
  File "/anaconda3/lib/python3.6/site-packages/setuptools/package_index.py", line 118, in distros_for_location
    wheel = Wheel(basename)
  File "/anaconda3/lib/python3.6/site-packages/setuptools/wheel.py", line 64, in __init__
    raise ValueError('invalid wheel name: %r' % filename)
ValueError: invalid wheel name: 'osmnet-0.1.0-py2.py3-any.whl'

Node Query Client Error

Description of the bug

I am trying to retrieve NYC's network using OSMnet and am running into a HTTP 400 error when replicating code from this

Full traceback:

File "/usr/local/lib/python2.7/site-packages/pandana/loaders/osm.py", line 183, in node_query
    lat_min, lng_min, lat_max, lng_max, tags=tags))
  File "/usr/local/lib/python2.7/site-packages/pandana/loaders/osm.py", line 115, in make_osm_query
    req.raise_for_status()
  File "/usr/local/lib/python2.7/site-packages/requests/models.py", line 939, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http://www.overpass-api.de/api/interpreter?data=%5Bout%3Ajson%5D%3B%28++node++%5B%22amenity%22%3D%22restaurant%22%5D++%2840.917577%2C40.477399%2C-73.700272%2C-74.25909%29%3B%29%3Bout%3B

OSM data (optional)

Bounding box data if it helps:

bbox = [40.917577, 40.477399, -73.700272, -74.259090] #lat-long bounding box for NYC

Environment

  • Operating system:
    OS Sierra
  • Python version:
    2.7
  • OSMnet version:
    0.1.5
  • OSMnet required packages versions (optional):

Paste the code that reproduces the issue here:

import pandana
from pandana.loaders import osm


bbox = [40.917577, 40.477399, -73.700272, -74.259090] #lat-long bounding box for NYC

osm_tags = '"amenity"="restaurant"'
pois = osm.node_query(bbox[0], bbox[1], bbox[2], bbox[3], tags=osm_tags)

Paste the error message (if applicable):

File "/usr/local/lib/python2.7/site-packages/pandana/loaders/osm.py", line 183, in node_query
    lat_min, lng_min, lat_max, lng_max, tags=tags))
  File "/usr/local/lib/python2.7/site-packages/pandana/loaders/osm.py", line 115, in make_osm_query
    req.raise_for_status()
  File "/usr/local/lib/python2.7/site-packages/requests/models.py", line 939, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http://www.overpass-api.de/api/interpreter?data=%5Bout%3Ajson%5D%3B%28++node++%5B%22amenity%22%3D%22restaurant%22%5D++%2840.917577%2C40.477399%2C-73.700272%2C-74.25909%29%3B%29%3Bout%3B

Geopandas 0.7 uses new object type for CRS

Description of the bug

Geopandas 0.7 changed the CRS type from a str to a pyproj.CRS class instance in this commit. This causes the check in

if (gdf.crs is not None) and ('proj' in gdf.crs) \
to fail with the below error.

Environment

  • Python version: 3.8

  • OSMnet version: 0.1.5

Paste the code that reproduces the issue here:

osm.pdna_network_from_bbox(lat_min=ymin, lng_min=xmin, lat_max=ymax, lng_max=xmax)

Paste the error message (if applicable):

  File "/usr/local/anaconda3/envs/nbtest6/lib/python3.8/site-packages/netbuffer/core/network.py", line 90, in get_osm_network
    network = osm.pdna_network_from_bbox(lat_min=ymin,
  File "/usr/local/anaconda3/envs/nbtest6/lib/python3.8/site-packages/pandana/loaders/osm.py", line 49, in pdna_network_from_bbox
    nodes, edges = network_from_bbox(lat_min=lat_min, lng_min=lng_min,
  File "/usr/local/anaconda3/envs/nbtest6/lib/python3.8/site-packages/osmnet/load.py", line 850, in network_from_bbox
    nodes, ways, waynodes = ways_in_bbox(
  File "/usr/local/anaconda3/envs/nbtest6/lib/python3.8/site-packages/osmnet/load.py", line 654, in ways_in_bbox
    osm_net_download(lat_max=lat_max, lat_min=lat_min, lng_min=lng_min,
  File "/usr/local/anaconda3/envs/nbtest6/lib/python3.8/site-packages/osmnet/load.py", line 136, in osm_net_download
    geometry_proj, crs_proj = project_geometry(polygon,
  File "/usr/local/anaconda3/envs/nbtest6/lib/python3.8/site-packages/osmnet/load.py", line 448, in project_geometry
    gdf_proj = project_gdf(gdf, to_latlong=to_latlong)
  File "/usr/local/anaconda3/envs/nbtest6/lib/python3.8/site-packages/osmnet/load.py", line 485, in project_gdf
    if (gdf.crs is not None) and ('proj' in gdf.crs) \
TypeError: argument of type 'CRS' is not iterable

Misc DeprecationWarning and FutureWarnings

Unit tests in Python 3.6.11 for osmnet v1.6.0 are returning misc DeprecationWarning and FutureWarnings when using geopandas 0.9, shapely 1.7.1, and pandas 1.1.0:

  1. DeprecationWarning: Flags not at the start of the expression '//(?s)(.*?)/' from https://github.com/UDST/osmnet/blob/dev/osmnet/load.py#L237, suggest to change (r'//(?s)(.*?)/', url)[0] to (r'(?s)//(.*?)/', url)[0]

  2. FutureWarning: Assigning CRS to a GeoDataFrame without a geometry column is now deprecated and will not be supported in the future. from: https://github.com/UDST/osmnet/blob/dev/osmnet/load.py#L445 suggest refactoring function to accept new schema.

  3. FutureWarning: '+init=<authority>:<code>' syntax is deprecated. '<authority>:<code>' is the preferred initialization method. When making the change, be mindful of axis order changes: https://pyproj4.github.io/pyproj/stable/gotchas.html#axis-order-changes-in-proj-6 return _prepare_from_string(" ".join(pjargs)) when using 'init' to initialize a projection, suggest using latest crs specification such as "EPSG:4326"

  4. FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead. import pandas.util.testing as pdt from: https://github.com/UDST/osmnet/blob/dev/osmnet/tests/test_load.py#L2 suggest removing use of pdt and instead use .equals()

Consider tagging a new release?

It would be nice to have some of the bug fixes since the latest version.

Explanation:
I was using pandana package which relies on osmnet, and am getting an error that is fixed by this commit, which was made after 0.6.1
6706683

Thanks for your time.

Suggested feature: custom OSM filters

Suggestion

User should be able to supply custom OSM filter when calling network_from_bbox

The two predefined filter choices for 'walk' and 'drive' are helpful, but somewhat limiting. Adding this would allow for much greater generalizability.

Network from bbox func should support either explicit lat/lons or bbox tuple - not both

Referring to the keyword arguments in this function (network_from_bbox): https://github.com/UDST/osmnet/blob/master/osmnet/load.py#L741

I can appreciate the desire to support inclusion of keyword arguments for the bounding box in more than one format, but in this case it seems (in my opinion) that the "downstream" costs are greater than any gains in flexibility. Specifically, one has to dig all the way to this check statement to realize that the reason why the lat/lon options are there is to be an alternative to the bbox.

This ultimately is the opinion of me as a user, but clarity in what arguments a function can take (in this case) supersedes the flexibility gains of allows us to input in either format.

I'd propose dropping support for lat/lon being submitted separately and enforce submission by the standard bbox 4-length tuple.

In other libraries that have employed this, such as this function, we can see how this function is negatively impacting the clarity of associated functions and creating bloated argument parameters.

My initial instinct as a user was to attempt to supply this (and be confused):

urbanaccess.osm.load.ua_network_from_bbox(
                                    bbox[0],
                                    bbox[1],
                                    bbox[2],
                                    bbox[3],
                                    bbox,
                                    network_type='walk',
                                    timeout=180,
                                    memory=None,
                                    max_query_area_size=2500000000L,
                                    remove_lcn=True)

A more ideal situation would be to just enforce the tuple format such that the supplied arguments would look akin to:

urbanaccess.osm.load.ua_network_from_bbox(
                                    bbox,
                                    network_type='walk',
                                    timeout=180,
                                    memory=None,
                                    max_query_area_size=2500000000L,
                                    remove_lcn=True)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.