rikkigibson/corvallis-bus-server

A Web API for the Corvallis Transit System.

corvallis-bus-server's Issues

Sunday schedule shown early Monday morning

Measure serial and parallel calls

AsParallel() is great, but it does incur a significant setup cost. Running the ETA lookup in parallel for small numbers of stops is silly.

Emit "over 30 minutes" value from /arrivals-summary and /favorites

The server currently has a "grace period" behavior where scheduled times between 20-30 minutes in the future are rendered as though they are estimates, i.e. instead of a timestamp like "2:46 PM" it'll be "22 minutes". This is intended to prevent jarring situations where a scheduled time draws closer and... abruptly disappears, because the scheduled time is too close to the present time and no estimate corresponding to that scheduled time has appeared.

A more intelligent way to communicate this situation to users would be to emit a message saying the bus will arrive in "over 30 minutes" during that grace period. This accurately conveys the fact that an estimate is optimistically expected to come in, but that the bus is running late (as it sometimes does...)

The fact that all clients now are consuming straight view models makes it relatively convenient to make a change on the server-side to emit different descriptions of arrival times and have clients mindlessly display them.

Some thought is warranted as to how to structure representations of the bus schedule internally to describe these various possible values i.e. a struct with a relative time on it and an IsSchedule/IsEstimate flag which can direct logic which produces the user-friendly descriptions of those arrival times.

Get rid of generated models

The generated XML models are making it so when I refactor the name of a namespace I get literally 3000 errors. Their usage should be replaced with some LINQ to XML in order to traverse the Connexionz data.

Use domain objects instead of dictionaries directly

It's supremely confusing what a type like Dictionary<int, Dictionary<string>, List<int>> means. I don't remember if that's the exact type used in here, but we have similar stuff.

Instead we should have named types with methods i.e. "GetRouteArrivalTimes(int stopId)"

Corvallis Bus App Issue

Hi, so couple days ago I tried using the bus app, it wasn't working at all and keeps saying failed to load, service updates also stopped working few months ago but thought it was my phone, Now the app or shortcut don't work at all for my iPhone or iPod, its Data/Cellular connection is fine because I can load other apps or transit sites, but Corvallis Bus App won't load.

Any chance this issue could be checked/looked at?
Thank you, hoping this can be fixed soon.

Enable CORS

Would allow web apps on other domains to make use of the API. Lack of CORS made authoring a Swagger difficult because the web-based editor can't make example calls to the API.

https://docs.microsoft.com/en-us/aspnet/core/security/cors?view=aspnetcore-2.1

Remove lazy loading from ConnexionzClient and GoogleTransitClient

The lazy loaded/cached properties on these clients necessitate recycling the app pool before performing an init task with new data. The methods that generate the data are only called during the init task, and we should just make sure to not call them redundantly.

Automatically run the init task at regular intervals

It's necessary for the long-term maintainability of this service for the init task to run automatically when transit data changes. We need to make that happen. It may be a matter of dispatching a task in the background when the first request of a given day comes in. Route info changes a bit in advance of when the changes in service occur, so it's unlikely to misinform that particular user who kicked off the re-run of the task.

AsParallel() vs Just spawning off tasks

Investigate which is faster since the work is I/O bound and not CPU bound

Fix stop sorting such that estimates are always shown before scheduled times

In this screenshot, route 2 is ordered before route 5 because the scheduled time backing that "over 30 minutes" message is nearer in the future than route 5's estimate. When estimates finally come in corresponding to a certain scheduled time, they're typically for 30 minutes in the future. This should probably be corrected on the server side so that all estimates are shown before any scheduled times.

Migrate to Web API

Currently this is an MVC site with a Web API attached. For the purposes of learning cross-platform development and more Azure stuff, the code here should get migrated into a Web API project and have all web app stuff stripped away. It should then be re-hosted as an Azure backend service, which will probably speed things up on that end, too.

Always return JSON in Web API 2

For simplicity, the app should always return JSON responses no matter what's in the request header. The following Stack Overflow answer describes how to do it.

http://stackoverflow.com/questions/12629144/how-to-force-asp-net-web-api-to-always-return-json

Bus app isn't loading bus times again

Hi, I made a post about a month or so ago and the app wasn't working, again it isn't working on any of my devices and was curious if you could take a look again at it. Other people are also having this issue I believe, many people were trying to refresh the app at the bus station.
Hopefully it'll get fixed soon.
Thank you

Scraping service alerts website is broken

Related to #30

The api/service-alerts endpoint used to work by scraping the HTML served up by the city. It still works locally but stopped working in Azure at some point. Requests to Connexionz are still working fine so I'm not sure what changed.

For now I published a workaround which just returns a dummy service alert that links to the website so the user can go there directly.

Disambiguate names of bus stops by direction

A user pointed out that there are multiple bus stops with the same name in the system, making information in the favorites list appear ambiguous. One of the better solutions for this would be to identify each set of identically-named stops and disambiguate them by the direction of travel, i.e. N/S/E/W/NE/NW/SE/SW. This could be done as part of the init task.

Fix off-by-one-minute errors

There are frequent off-by-one errors that (probably) result from converting the scheduled time from DateTimeOffset to "minutes-from-now" and back to DateTimeOffset. Let's just change the internal representation so that it can always hold a DateTimeOffset for scheduled times and no such lossy conversion will be necessary.

Move from Azure Scheduler jobs to Logic Apps

https://docs.microsoft.com/en-us/azure/scheduler/migrate-from-scheduler-to-logic-apps

Add support for multi-route directions

It would be super convenient to have an endpoint that takes the users current location coordinate and the destination coordinate as input and returns JSON data that includes:

• All bus routes the user will need to take to get to their destination
• For each route the user will take on the trip, which stop should they get on and off

I'm sure I may be missing something here and I am aware that this is not the easiest and quickest thing to implement, but I feel adding this capability to the API will take it to the next level, allowing developers to really build something robust.

Google Transit zipfile should be stored with project

Seems wrong to depend on a dropbox link working indefinitely. Maybe embed it as a resource in CorvallisBus.Core?

https://github.com/RikkiGibson/Corvallis-Bus-Server/tree/db07122b16442e8e5adf17a323e3570c5993efd5/CorvallisBus.Core/WebClients/GoogleTransitClient.cs#L194

Write more Connexionz deserialization tests

Connexionz XML changes periodically. We should have tests which include the XML we expect and deserialize to the appropriate entity. Then, when the XML changes, we will be able to compare the before/after data and decide the appropriate change to make to the models.

This work is started in CorvallisBus.Test/DeserializationTests.cs

Add support for military time

Currently the times are sent to the clients in 12-hour time, regardless of the client's device settings.

Move data dependency to GTFS as much as possible

Some of the data is ingested by reading Platform.rxml and RoutePattern.rxml. But it should probably change to get the same data via files in the GTFS archive.

This might make it easier to make this app start showing transit data for a new transit system (i.e. King County).

Validation issues for route 6 and route 7

This could mean we're not importing the data correctly.

 CorvallisBus.Test.TransitClientTests.ValidateInitJob
   Source: TransitClientTests.cs line 30
   Duration: 640 ms

  Standard Output: 
Route 7 at stop 10723 has an ordering discrepancy in its schedule. Arrival time 06:32:07 is followed by 06:32:07
Route 7 at stop 10845 has an ordering discrepancy in its schedule. Arrival time 07:05:39 is followed by 07:05:39
Route 7 at stop 10850 has an ordering discrepancy in its schedule. Arrival time 07:06:24 is followed by 07:06:24
Route 7 at stop 10543 has an ordering discrepancy in its schedule. Arrival time 06:42:36 is followed by 06:42:36
Route 7 at stop 10536 has an ordering discrepancy in its schedule. Arrival time 06:43:27 is followed by 06:43:27
Route 7 at stop 11186 has an ordering discrepancy in its schedule. Arrival time 06:48:43 is followed by 06:48:43
Route 7 at stop 11172 has an ordering discrepancy in its schedule. Arrival time 06:49:30 is followed by 06:49:30
Route 7 at stop 11164 has an ordering discrepancy in its schedule. Arrival time 06:50:00 is followed by 06:50:00
Route 7 at stop 11070 has an ordering discrepancy in its schedule. Arrival time 06:51:33 is followed by 06:51:33
Route 7 at stop 11062 has an ordering discrepancy in its schedule. Arrival time 06:52:13 is followed by 06:52:13
Route 7 at stop 11058 has an ordering discrepancy in its schedule. Arrival time 06:58:16 is followed by 06:58:16
Route 7 at stop 11043 has an ordering discrepancy in its schedule. Arrival time 06:59:05 is followed by 06:59:05
Route 7 at stop 11036 has an ordering discrepancy in its schedule. Arrival time 07:00:00 is followed by 07:00:00
Route 7 at stop 11027 has an ordering discrepancy in its schedule. Arrival time 07:00:40 is followed by 07:00:40
Route 7 at stop 11155 has an ordering discrepancy in its schedule. Arrival time 07:01:41 is followed by 07:01:41
Route 7 at stop 11140 has an ordering discrepancy in its schedule. Arrival time 07:02:13 is followed by 07:02:13
Route 7 at stop 11138 has an ordering discrepancy in its schedule. Arrival time 07:03:14 is followed by 07:03:14
Route 7 at stop 11089 has an ordering discrepancy in its schedule. Arrival time 06:45:33 is followed by 06:45:33
Route 7 at stop 12669 has an ordering discrepancy in its schedule. Arrival time 06:46:28 is followed by 06:46:28
Route 7 at stop 12653 has an ordering discrepancy in its schedule. Arrival time 06:47:11 is followed by 06:47:11
Route 7 at stop 12648 has an ordering discrepancy in its schedule. Arrival time 06:48:18 is followed by 06:48:18
Route 7 at stop 12630 has an ordering discrepancy in its schedule. Arrival time 07:09:01 is followed by 07:09:01
Route 7 at stop 12624 has an ordering discrepancy in its schedule. Arrival time 07:10:00 is followed by 07:10:00
Route 7 at stop 12611 has an ordering discrepancy in its schedule. Arrival time 07:11:01 is followed by 07:11:01
Route 7 at stop 12607 has an ordering discrepancy in its schedule. Arrival time 07:11:28 is followed by 07:11:28
Route 7 at stop 12598 has an ordering discrepancy in its schedule. Arrival time 07:11:58 is followed by 07:11:58
Route 7 at stop 12579 has an ordering discrepancy in its schedule. Arrival time 07:12:45 is followed by 07:12:45
Route 7 at stop 12682 has an ordering discrepancy in its schedule. Arrival time 06:36:59 is followed by 06:36:59
Route 7 at stop 12676 has an ordering discrepancy in its schedule. Arrival time 06:37:44 is followed by 06:37:44
Route 7 at stop 12567 has an ordering discrepancy in its schedule. Arrival time 06:38:43 is followed by 06:38:43
Route 7 at stop 12551 has an ordering discrepancy in its schedule. Arrival time 06:39:16 is followed by 06:39:16
Route 7 at stop 12546 has an ordering discrepancy in its schedule. Arrival time 06:40:00 is followed by 06:40:00
Route 7 at stop 10562 has an ordering discrepancy in its schedule. Arrival time 06:55:00 is followed by 06:55:00
Route 7 at stop 13254 has an ordering discrepancy in its schedule. Arrival time 06:31:32 is followed by 06:31:32
Route 7 at stop 12477 has an ordering discrepancy in its schedule. Arrival time 06:32:29 is followed by 06:32:29
Route 7 at stop 12465 has an ordering discrepancy in its schedule. Arrival time 06:32:58 is followed by 06:32:58
Route 7 at stop 12454 has an ordering discrepancy in its schedule. Arrival time 06:33:44 is followed by 06:33:44
Route 7 at stop 12449 has an ordering discrepancy in its schedule. Arrival time 06:34:16 is followed by 06:34:16
Route 7 at stop 12420 has an ordering discrepancy in its schedule. Arrival time 06:35:00 is followed by 06:35:00
Route 7 at stop 12412 has an ordering discrepancy in its schedule. Arrival time 06:35:35 is followed by 06:35:35
Route 7 at stop 12408 has an ordering discrepancy in its schedule. Arrival time 06:36:13 is followed by 06:36:13
Route 7 at stop 12393 has an ordering discrepancy in its schedule. Arrival time 07:13:13 is followed by 07:13:13
Route 7 at stop 10246 has an ordering discrepancy in its schedule. Arrival time 07:13:59 is followed by 07:13:59
Route 7 at stop 10251 has an ordering discrepancy in its schedule. Arrival time 07:14:27 is followed by 07:14:27
Route 7 at stop 10267 has an ordering discrepancy in its schedule. Arrival time 07:15:00 is followed by 07:15:00
Route 7 at stop 10279 has an ordering discrepancy in its schedule. Arrival time 07:16:11 is followed by 07:16:11
Route 7 at stop 10298 has an ordering discrepancy in its schedule. Arrival time 07:18:17 is followed by 07:18:17
Route 7 at stop 14421 has an ordering discrepancy in its schedule. Arrival time 07:04:16 is followed by 07:04:16
Route 7 at stop 14432 has an ordering discrepancy in its schedule. Arrival time 07:05:00 is followed by 07:05:00
Route 7 at stop 14292 has an ordering discrepancy in its schedule. Arrival time 06:44:03 is followed by 06:44:03
Route 7 at stop 14285 has an ordering discrepancy in its schedule. Arrival time 06:45:00 is followed by 06:45:00
Route 7 at stop 11418 has an ordering discrepancy in its schedule. Arrival time 06:41:21 is followed by 06:41:21
Route 7 at stop 14886 has an ordering discrepancy in its schedule. Arrival time 06:42:02 is followed by 06:42:02
Route 7 at stop 14990 has an ordering discrepancy in its schedule. Arrival time 06:40:54 is followed by 06:40:54
Route 7 at stop 14903 has an ordering discrepancy in its schedule. Arrival time 07:07:34 is followed by 07:07:34
Route 7 at stop 14840 has an ordering discrepancy in its schedule. Arrival time 07:08:19 is followed by 07:08:19
Route 7 at stop 14864 has an ordering discrepancy in its schedule. Arrival time 06:30:00 is followed by 06:30:00
Warning: 6 does not have the same number of arrivals at all stops. Stop 12973 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 12960 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 12956 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 12939 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 12941 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 12925 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 13055 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 12034 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 12023 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 12010 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 12006 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 11983 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 11965 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 14216 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 11954 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 11949 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 11931 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 13188 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 13212 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 13231 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 10723 has 30 arrivals while stop ID 14864 has 60 arrivals.
Warning: 6 does not have the same number of arrivals at all stops. Stop 10298 has 30 arrivals while stop ID 14864 has 60 arrivals.

Validate results after init job

There should be a unit test that runs at least the deterministic parts of the "init job", validates a few invariants, and fails if those invariants are violated. For example, stops that are later on in a route path should not have earlier arrival times :)

Add /stops/ endpoint To API

It would be really convenient to have the API include a /stops/ endpoint to return latitude and longitude information for each stopID provided.

The new endpoint could respond with a similar json response to the /favorites/ endpoint or the static endpoint, but include latitude/longitude info for the stop as well. Maybe /favorites/ could also return lat/long info on stopID's to avoid hitting the new endpoint on top of /favorites/ if you did need the lat/long data.

.NET Core/Azure

The Gearhost and .NET Framework 4.5 setup is getting a little long in the tooth--would be nice to move this over to .NET Core 2.0, maybe on Azure since I think I have a bunch of credit I can use.

@jschneidereit may wish to assist here :)