esri/public-transit-tools

Tools for working with GTFS public transit data in ArcGIS

public-transit-tools's Issues

Add GTFS: Clarify debugging steps in Troubleshooting Guide

There is some confusion over the phrase "Pick a transit line" and unfortunate terminology clashes with the words "Route" and "Stop" since they could apply equally to transit stuff or to NA stuff.

Fix Add GTFS unicode encoding error message

[Add GTFS to a Network Dataset]
The unicode encoding error message thrown by sqlize_csv.py lists the wrong GTFS file.

BetterBusBuffers: Count Trips at Points Online: Use concurrent.futures to parallelize calls to the service

AddGTFStoaNetworkDataset: Investigate slowness in Copy Traversed Source Features for a large number of routes

According to a user, Copy Traversed Source Features (with Transit) starts to get slower for large numbers of routes. The number of routes processed per second goes down.

Check for duplicate trip_id values in trips.txt

[Add GTFS to a Network Dataset]
GTFS data from TheBus had 114 rows in trips.txt with the same trip_id value. This is an invalid GTFS dataset, but my tool needs to fail with a helpful error message explaining why this won't work. As it is, the error doesn't get caught until caching time when it complains that an item with that key has already been added to the trips table, not something a user will understand.

Add check for headway values of 0 in frequencies.txt

If frequencies.txt has a value of 0 for headway, it causes errors in Add GTFS step 1.

GDB gets corrupted if you try to reuse it

When you run '1) Generate Transit Lines and Stops' tool for the second time using the same file gdb, it fails with the error message – Failed to execute (CreateFeatureclass). This happens even if the outputs (Stops, TransitLines) or the entire featureDataset from the previous instance have been deleted.

GetEIDs fails validation if input is a network dataset layer

It throws "The feature dataset where your network dataset resides does not have a feature class called 'TransitLines'. Please use a valid network dataset." It's getting confused because when the input is a network dataset layer rather than the catalog path, it isn't correctly finding the feature dataset. Need special handling for this case.

GenerateGTFSShapes: Create a version that uses ArcGIS Online routing services instead of local data

This will charge credits but won't require local data.
The current hold-up is that the AGOL service returns TRUE_LINES_WITHOUT_MEASURES, and the vertices of the returned features without measures are not guaranteed to be in the correct order, so the shapes get messed up.

Add tool to count trips on lines/corridors

BetterBusBuffers currently offers the ability to count trips visiting specific stops or within a short walk distance of areas or points of interest. Multiple users have asked me for the ability to count trips on the transit lines themselves so they can symbolize the lines based on frequency of service.

Considerations:

• There are many cases where multiple routes run along the same corridor between the same two stops. Should all these to be summed together when considering the frequency of service along that corridor, or should each bus/subway line counted separately? Or both?
• Where should I obtain the line geometry? Using shapes is difficult because they aren't directly related to the stops (and shapes.txt is optional and even then often not very good), but connecting stops with straight lines doesn't look nice or accurate.
• Should default symbology be based on color or make use of line width a la Jarrett Walker?

DisplayGTFSRouteShapes: Add a workaround for when shape_ids don't match in trips.txt

The Montreal data was having this problem a while ago.

Add GTFS: Investigate possible issues with non-English OS/region & language settings

A user had a problem with "Use specific dates" and a Spanish OS, and the issue was resolved when he switched his region and language settings to US/English. Could be user error, but it could be a bug. The NA layer settings dialog should handle this properly, but perhaps there's an issue in the transit evaluator code.

sqlite3 replace c.fetchall() with c.fetchone() to conserve memory

The fetchall() thingie in sqlite3 returns the entire set of query results into a variable, which means into memory. This can cause memory issues for larger queries. Unless there's a reason I need the entire set of results at once, I can replace the following pattern:

c.execute(<blah>) results = c.fetchall() for thing in results: <do stuff>

with

c.execute(<blah>) for thing in c: <do stuff>

(Code formatting is messed up, but you get the picture.)

• Add GTFS to a Network Dataset
• BetterBusBuffers
• Display GTFS in ArcGIS
• Generate GTFS Shapes
• Edit GTFS Stop Locations (Nothing to do here)
• Interpolate Blank Stop Times (Leaving most cursors as is because they don't lend themselves well to a one-by-one iteration.

Improve route estimates using NA Navigation fields

Use the Bearing and BearingTol fields in the NA route layer to improve the estimated on-street routes calculated in Generate GTFS Shapes Step 1. Transit stops tend to locate on side streets, causing the estimated route to enter the side street, make a u-turn, and then continue along the main street. If I can first connect all the stops using straight lines and use those to estimate the bearing (the general direction of travel), then I can use the Bearing and BearingTol fields in the NA layer to influence locating behavior, and hopefully those stops will get located on the main road where they belong.

I don't know if this will work, but it's worth a try.

Stop names

Adding stop_names when running Generate GTFS Shapes with ArcGIS Online or Network Analyst would be very welcome. Would it be possible to add such functionality?

Interpolate Blank Stop Times: Make interpolation method that uses network travel time for estimates

It might be nice to estimate arrival_time and departure_time values for stops using something more complex and "accurate" than simple interpolation. I would like to implement something that uses the road network to estimate travel time between stops.

Are people interested in this enhancement? Or is the simple interpolation method good enough for your purposes?

Improve memory management in Calculate Accessibility Matrix

Possibly Pandas can be used for this, but then it's not backwards-compatible in older version of ArcGIS. Look into this more as time allows.

BetterBusBuffers: Add ability to use different buffer distances for different modes

Stops aren't necessarily mode-specific, so this might be challenging.

Add .chm help files to all tools

Make shapes.txt file from existing feature class

People occasionally ask me if they can generate a GTFS shapes.txt file from an existing feature class of route shapes.

The main challenge is that the feature class needs to have a unique shape for each unique sequence of stops, and lots of logic will be needed to ensure that their input feature class is correct for this.

Add support for transfers.txt

Currently Add GTFS to a Network Dataset does nothing with transfers.txt. I need a good sample dataset that uses this file effectively before I can integrate support for it.

Determine if pandas is a valid replacement for sqlite in some/all of my tools

Need to run some tests on speed and memory management to see if pandas dataframes might be a better way to work with GTFS data than my sqlite databases.

Create tool to symbolize route frequency using line width

Symbolize transit lines in the map based on frequency of service, a la Jarrett Walker

Errors with all tools

Hi

Can these tools run in ArcGIS 10.4.1?

I have an ImportError: C extenstion: no module named builtins not built.

Thanks

GenerateGTFSShapes: Add ability to create a subset of shapes

Maybe the user wants to update just one shape.

Add timepoint field with value of 0 to indicate interpolation

In the output of Interpolate Blank Stop Times, add/update the timepoint field in stop_times.txt. Give all interpolated stop_times a value of 0 for timepoint, following the GTFS specification.

Resolve inconsistent results from transit evaluator

Sometimes midway through a transit trip, the traveler misses the next bus by a small fraction of a second. It appears to be the result of an accumulation of small rounding errors resulting from the way the NA solvers account for non-discreet positions along the network. It's actually a more fundamental Network Analyst issue, not a transit evaluator problem, but I need to find a workaround.

Make a tool to edit existing GTFS shapes

Feed: 608 FC: CTAK, Juneau Alaska, GTFS data issues.

Display GTFS in ArcGIS

I am working on ArcMap 10.1 and I am trying to use Display GTFS in ArcGIS, but then error is occurring and its about some module. Could anyone please check if we can solve it or has it already been solved? Also, could anyone tell me what is the logic behind getting the routes shapefile.

Enhance Count Trips on Lines to use actual shapes

Count Trips on Lines currently just generates a straight line between connected stops, ignoring the actual paths the transit vehicles travel, so it isn't great for visualization. Enhance the tool to use shapes.

This will be challenging because shapes are only loosely tied to the stops, and the code would have to have a lot of data validation and checking for invalid shapes. I could theoretically rely on the shape_dist_traveled field in stop_times and use some linear referencing to assume the position of the stop along the shape, but this seems pretty prone to data quality issues. Plus, how to handle corridors served by trips with different shapes?

Create a blank stop_time removal utility

Many GTFS datasets contain blank values for arrival_time and departure_time in stop_times.txt. This can be intentional (datasets using time points) or by mistake. Because no exact values are present, these datasets cannot be used in my tools. It is possible to interpolate stop_time values as a reasonable estimate, but interpolation needs to be done with care because the analysis results based on this interpolation could produce incorrect or misleading results.

DisplayGTFSRouteShapes: Make this tool output stops as well

It might be nice to change this tool to something like "DisplayGTFSDataInMap" and have it output not only shapes but also the stops.

Make a better installer

@matt3274 says he knows of a better way to register transitevaluator.dll with ArcMap. This is a real problem for a lot of users.

Unable to create on-street Routes because the Solve failed

Hi everyone,

i'm trying to generate shapes with network analysis, however, when I'm running the tool, it gave me these comments per each route:

Unable to create on-street Routes because the Solve failed.
Solve warning messages:
Cannot use hierarchy when the hierarchy level count is zero.

Solve error messages:
ERROR 030024: Solve returned a failure.
Invalid context.
Failed to execute (Solve).

Could anyone help me to solve this problem?
Thanks!

GeoEvent Adaptor for GTFS realtime data

It would be great to have a GeoEvent Adaptor for GTFS realtime data. I've looked around but have not found anything.

Make tool to calculate ADA/paratransit "service area" requirement

I need to research the FTA regulation, but I've been told by transit agency representatives that it's based on simple buffers around fixed-route transit lines. The agency is required to provide on-demand service for disabled and elderly citizens in the area within a certain distance of transit lines (even though that doesn't make much sense), and there are certain rules about holes within the coverage area, and they can make certain exceptions for different types of routes.

Make tool to automate Service Area time lapse data prep

The tool should probably take in a Service Area layer, solve the layer in a loop over a user-specified time window, and spit out a feature class with time fields. Further instructions will show how to create the time lapse video in ArcMap or Pro.

AddGTFStoaNetworkDataset: Handle case when trips.txt has a route_id that isn't in routes.txt

BetterBusBuffers: Improve method for counting overlapping polygons

The current method is slow and clunky and doesn't actually work all that well.

Add support for parent_stations

In Add GTFS to a Network Dataset, street connectors should connect to parents stations, and child stations should connect only to the parent stations and not to the street. I need a good dataset that uses this effectively to use for testing.

Make a tool to assess schedule adherence

Collect gtfs-rt over a period of time and compare it with the schedule data to highlight areas where the schedule needs work (and possibly export a corrected schedule back out to GTFS). Let's close the feedback loop!

Do you want a tool like this?
If you work for a transit agency and are interested in studying schedule adherence, I would love to talk to you. Please contact me.

Is there such a thing as open passenger count data, and where can I find some?

I'd love to play around with some passenger count data, but I don't know of any sources of open passenger count data, and I've heard horror stories from a few people who jumped through numerous hoops (including FOIA requests) trying to get it.

Add GTFS: Figure out why network drives don't work

You can't store the network dataset on a network drive. The evaluator doesn't like it. Why?

BetterBusBuffers: Count Trips at Points Online: Consider smarter chunking methods

The goal is to minimize the total number of OD pairs we have to calculate.

Fix error in Calculate Accessibility Matrix tool

User used the same layer for both Origins and Destinations.

1900-01-03 08:58:00
1900-01-03 08:59:00
1900-01-03 09:00:00
Calculating statistics and writing results...
Failed script CalculateAccessibilityMatrix...

Traceback (most recent call last):
File "E:\Repositories\CMTA_GIS_Tools\Admin Tools\GTFS_Transit_Tools\AddGTFStoaNetworkDataset_0_5_4_0\scripts\CalculateAccessibility.py", line 213, in
for dest in OD_count_dict[origin_OID]:
KeyError: 2536

Failed to execute (CalculateAccessibilityMatrix).

Replace folder name with agency_name from agency.txt

Currently tools that accept multiple GTFS datatsets prepend a dataset designator to data fields in the sql database and in the output to avoid clashes between things like stop_id and trip_id values used by multiple agencies. The designator is currently the name of the folder that the GTFS dataset was in when you ran the tool. Replace this with the agency_name from agency.txt (but still check for clashes).

Status Update on ArcGIS Pro Integration

Is there any status update on when and how network analyst integration in ArcGIS Pro will happen?

I saw that the .NET framework API for Pro was released, and so I was curious about what the bottlenecks on this integration are?

AddGTFStoaNetworkDataset: Improve memory management in step 1

The stop_times.txt file is often huge, and you either have to read it all into memory or write it to a query-able sql database so you can figure out which stop times belong to which trip. The input file is not required to be ordered, so you can't make any assumptions about whether you've found adjacent stops until you've read in the entire file. The current strategy fails for really large datasets, like De Lijn.

Make tool to split up large GTFS datasets into smaller pieces

Large GTFS datasets (such as the ones for entire countries in Europe) can be unwieldy, and I've had a request for a tool to split them up and extract just the relevant area for a particular study.