wsdot-gis/wsdot-traffic-gp

Provides information from the WSDOT Traveler Information API to ArcGIS Software via Geoprocessing Scripts

wsdot-traffic-gp

The scripts in this repository can be used to consume WSDOT Traveler Information API REST endpoints in ArcGIS software.

Setup

Install package

To install this package onto your computer using pip, use the following command. For more information on installing packages, see the Installing Packages tutorial from the PyPA Python Packaging User Guide.

pip install wsdottraffic

If you are running this command from PowerShell, you will need to have administrator privileges.

Start-Process pip "install wsdottraffic" -Verb RunAs

Default access code

You can set a default access code, so you don't need to provide it via function parameter or script argument, by setting an environment variable called WSDOT_TRAFFIC_API_CODE to the default access code.

Modules

See the modules' docstrings for more details on how to use the scripts.

wsdottraffic

This module provides the ability to query the REST endpoints and return the results as a dictionary.

• Note that this script has no ArcGIS dependencies and can be run without any ArcGIS software installed.
• Should run in either v2.7+ or v3.5.2+ of Python.

wsdottraffic.gp

Consume the REST endpoints and return the results as a file geodatabase.

• Requires ArcPy
• Should work with either ArcGIS Desktop or ArcGIS Pro, and the versions of Python that they come with.

Scripts

You can get help for any of the scripts using the -h argument.

python -m wsdottraffic -h

wsdottraffic.gp / wsdottrafficgp

• Downloads data from API
• Creates feature class and tables if not already existing
• If tables exist, truncates them
• Inserts the data into feature classes and tables inside of a file geodatabase.
• Zips the file geodatabase (which is a folder w/ .gdb extension).

python -m wsdottraffic.gp.creategdb

or

wsdottrafficgp

wsdottraffic.dumpjson / wsdottraffic

Downloads data from API and exports JSON files: one with the data and one with automatically detected field definitions.

python -m wsdottraffic

or

wsdottraffic

wsdottraffic.gp.multipointtopoint / multipointtopoint

Calls the Multipart to Singlepart tool for each multipoint feature class in a geodatabase. Added feature classes will have the same name as its source, but with the added suffix _singlepart.

python -m wsdottraffic.gp.multipointtopoint YourGDBNameHere.gdb

or

multipointtosinglepoint YourGDBNameHere.gdb

Notes for developers

The example below shows how to run one of the scripts from within the project directory

python -m src.wsdottraffic.gp.creategdb

Unit tests (test_*.py)

These are test scripts for use with the unittest Python module.

PowerShell scripts for developers

The PowerShell scripts are intended for use by developers working on this project and are not used by consumers of the library. Use the Get-Help command for more info on these scripts.

wsdot-traffic-gp's Issues

Add support for Bridge Clearances

A Bridge Clearances endpoint was added after these tools were created. Add support for the Bridge Clearances endpoint.

Add ability for Scanweb table creation to use templates

Scanweb table creation is handled by a different module from the rest, and does not currently support creating new table using templates (which is faster than using arcpy.AddField).

Standardize output GIS field names

Modify the script so that the output properties / field names used for GIS purposes (e.g., Longitude, Latitude, RoadName (both for points and lines, with start and end points)) are standardized across data sets.

Add support for Weather Information

File size keeps increasing after each truncate+update

...the file size eventually gets so large that uploading via overwrite-hosted-features fails.

Create symbology for output layers

Required

• Commercial Vehicle Restrictions
• Highway Alerts
• Highway Cameras
• Mountain Pass Conditions
• Traffic Flow
• Weather Information

Optional

The following aren't required for 511 project

• Border Crossings
• Bridge Clearance
• Toll Rates
• Weather Stations

Modify to use arcpy.da

Modify travelerinfogp.py to use arcpy.da when populating tables for better performance.

Scanweb data GeoJSON serialization fails

Investigate ligeos Python package for linear referencing

Could linear referencing be performed faster than with arcpy using ligeos?

Upgrade toolbox to Python Toolbox

Convert the binary toolbox (.tbx) into a Python Toolbox (.pyt).

Add unit tests

Feature classes do not contain features

In the agol-upload branch, some tables and feature classes are failing when attempting to add data to them. Appears to be related to date fields. Sometimes error messages about an invalid length are returned.

Add support for Weather Stations

GP tools should output Feature Classes instead of regular Tables

• Be sure to specify WGS84 spatial reference on feature classes!
• Data representing points should be returned as Point feature classes.
• Data representing line segments should be returned as Polyline(M?) feature classes.
  • ArmCalc'ed (begin and end points + route name)
  • Located on route. Route layer provided via GP parameter.
  • Some of this data only has a begin point (and a 0 endpoint). This data should be separated into a point feature class.

Add script to upload File GDB to ArcGIS Online and update

• Add ability to upload zipped file geodatabase file to ArcGIS Online
• After uploading zipped file GDB, publish that uploaded file as a feature service.
• Add ability to update data in existing feature service.
• Change the login method to use Windows Account (Active Directory, SAML, STS) instead of the current method of reading user name and password from a JSON file. (Note that JSON file is not currently stored in this repository.)

JSON parsing issue

Occurs when attempting to detect an invalid route name in one of the data fields. Occurs at https://github.com/WSDOT-GIS/wsdot-traffic-gp/blob/master/src/wsdot/traffic/jsonhelpers.py#L85

  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Scripts\createwsdottrafficgdb-script.py", line 11, in
<module>
    load_entry_point('wsdot.traffic==1.1.0', 'console_scripts', 'createwsdottrafficgdb')()
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\wsdot\traffic\gp\creategdb.py", line 62, in main
    create_gdb(args.gdb_path, args.code, None, names)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\wsdot\traffic\gp\creategdb.py", line 90, in create_gdb
    data = get_traveler_info(name, access_code)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\wsdot\traffic\__init__.py", line 74, in get_traveler_info
    object_hook=parse_traveler_info_object)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\json\__init__.py", line 332, in loads
    return cls(**kw).decode(s)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\json\decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\json\decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\wsdot\traffic\jsonhelpers.py", line 85, in parse_traveler_info_object
    bad_route_name.match(val[roadway_location_key])):
TypeError: expected string or bytes-like object
C:\Users\jacksoe\Source\Repos\wsdot511\Update-511.ps1 : Failed to create or update file geodatabase.
At line:1 char:1

Add support for tolling