This project forked from ibi-group/datatools-server
Server for Conveyal's GTFS data management platform. NOTE: Active development has returned to https://github.com/conveyal/datatools-server
License: MIT License
The core application for Conveyal's transit data tools suite.
View the latest documentation at ReadTheDocs.
Note: dev branch docs can be found here.
For the project-updated subscription, there is no email notification scheduled for subscribed users.
Editor feed DBs become corrupted for some unknown reason. One guess is that a shutdown signal is causing the MapDB to corrupt, but this should be handled by the DBMaker#closeOnJvmShutdown option.
Also, it appears that the scheduleExceptionCountByDate map is possibly tangled up in the corruption problem.
...
12:53:33.747 [pool-1-thread-1] INFO com.conveyal.gtfs.BaseGTFSCache - db files written.
12:53:52.178 [pool-1-thread-1] WARN c.c.d.m.c.api.GtfsApiController - New version found for gtfs/STIF-20170428T093236Z-6e67d5ff-7a7a-48b4-a9b1-528a9f73c68c.db is null. Downloading from s3...
12:53:52.464 [pool-1-thread-1] INFO com.conveyal.gtfs.BaseGTFSCache - Attempting to download cached GTFS MapDB.
12:53:52.525 [pool-1-thread-1] WARN com.conveyal.gtfs.BaseGTFSCache - DB file for key gtfs/STIF-20170428T093236Z-6e67d5ff-7a7a-48b4-a9b1-528a9f73c68c-db does not exist on S3.
12:53:52.526 [pool-1-thread-1] INFO com.conveyal.gtfs.BaseGTFSCache - Feed not found locally, downloading from S3.
12:53:52.949 [pool-1-thread-1] ERROR com.conveyal.gtfs.BaseGTFSCache - Could not download feed at s3://datacatalogue/gtfs/STIF-20170428T093236Z-6e67d5ff-7a7a-48b4-a9b1-528a9f73c68c-db.
12:53:52.949 [pool-1-thread-1] WARN c.c.d.m.c.api.GtfsApiController - New version found for gtfs/STIF-20170428T093236Z-6e67d5ff-7a7a-48b4-a9b1-528a9f73c68c.db.p is null. Downloading from s3...
12:53:59.120 [pool-1-thread-1] INFO com.conveyal.gtfs.BaseGTFSCache - Attempting to download cached GTFS MapDB.
12:53:59.160 [pool-1-thread-1] WARN com.conveyal.gtfs.BaseGTFSCache - DB file for key gtfs/STIF-20170428T093236Z-6e67d5ff-7a7a-48b4-a9b1-528a9f73c68c-db-p does not exist on S3.
12:53:59.160 [pool-1-thread-1] INFO com.conveyal.gtfs.BaseGTFSCache - Feed not found locally, downloading from S3.
12:53:59.167 [pool-1-thread-1] ERROR com.conveyal.gtfs.BaseGTFSCache - Could not download feed at s3://datacatalogue/gtfs/STIF-20170428T093236Z-6e67d5ff-7a7a-48b4-a9b1-528a9f73c68c-db-p.
12:53:59.168 [pool-1-thread-1] WARN c.c.d.m.c.api.GtfsApiController - New version found for gtfs/STIF-20170428T093236Z-6e67d5ff-7a7a-48b4-a9b1-528a9f73c68c.zip is null. Downloading from s3...
...
When updating the trip pattern's direction using the UI, the direction value does not appear to be updating properly. The root cause appears to be that the TripPatternSerializer::serialize method is not being called properly for some reason (this happens in Base.java).
Nevertheless, it's not entirely clear that we even need this serializer. There is no issue in just delivering these values to the client as A and B as they are specified in the TripDirection enum (just needs a corresponding fix on the client).
11:43:49.287 [qtp83546919-15] ERROR c.c.datatools.manager.DataManager - error
java.lang.NullPointerException: null
at com.conveyal.datatools.manager.controllers.api.UserController.getRecentActivity(UserController.java:215) ~[dt-6a27c88-dirty.jar:na]
at spark.ResponseTransformerRouteImpl$1.handle(ResponseTransformerRouteImpl.java:47) ~[dt-6a27c88-dirty.jar:na]
at spark.http.matching.Routes.execute(Routes.java:61) ~[dt-6a27c88-dirty.jar:na]
at spark.http.matching.MatcherFilter.doFilter(MatcherFilter.java:126) ~[dt-6a27c88-dirty.jar:na]
at spark.embeddedserver.jetty.JettyHandler.doHandle(JettyHandler.java:50) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:189) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.server.Server.handle(Server.java:517) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:242) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:75) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:213) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) ~[dt-6a27c88-dirty.jar:na]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_112]
11:43:49.288 [qtp83546919-15] INFO spark.http.matching.MatcherFilter - The requested route [/api/manager/secure/user/auth0|57a1f0c5b8155e90194e5087/recentactivity] has not been mapped in Spark
On mongo-pojo branch at commit number 2b91f71e98384f51e0d57d973a00e7c5731ab40a:
sept. 19 09:56:55.178 [pool-2-thread-1] INFO com.conveyal.osmlib.PBFInput - Done parsing PBF.
sept. 19 09:56:55.179 [pool-2-thread-1] INFO com.conveyal.osmlib.PBFInput - Read 0 nodes, 0 ways, 0 relations.
sept. 19 09:56:55.282 [pool-2-thread-1] INFO com.conveyal.r5.streets.StreetLayer - Making street edges from OSM ways...
sept. 19 09:56:55.293 [pool-2-thread-1] WARN c.c.d.manager.models.FeedVersion - Unknown error encountered while building network for TISSEO2-20170919T095640-02-2341da9d-c09c-48fc-8dbd-7e94c7683f0f.zip.
java.lang.ArrayIndexOutOfBoundsException: Attempt to seek beyond end of edge store
at com.conveyal.r5.streets.EdgeStore$Edge.seek(EdgeStore.java:348)
at com.conveyal.r5.streets.StreetLayer.loadFromOsm(StreetLayer.java:273)
at com.conveyal.r5.streets.StreetLayer.loadFromOsm(StreetLayer.java:170)
at com.conveyal.r5.transit.TransportNetwork.fromFiles(TransportNetwork.java:166)
at com.conveyal.r5.transit.TransportNetwork.fromFeeds(TransportNetwork.java:133)
at com.conveyal.datatools.manager.models.FeedVersion.buildTransportNetwork(FeedVersion.java:522)
at com.conveyal.datatools.manager.jobs.BuildTransportNetworkJob.run(BuildTransportNetworkJob.java:31)
at java.util.ArrayList.forEach(ArrayList.java:1249)
at com.conveyal.datatools.common.status.MonitorableJob.jobFinished(MonitorableJob.java:83)
at com.conveyal.datatools.manager.jobs.ValidateFeedJob.run(ValidateFeedJob.java:50)
at com.conveyal.datatools.manager.jobs.ProcessSingleFeedJob.run(ProcessSingleFeedJob.java:55)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
`
``
Am I required to serve the frontend from Amazon's S3 Bucket?
I know I can avoid using S3 for the gtfs cache configuring application.data.use_s3_storage to false at datatools-server:server.yml. But when it comes to the assets I only found application.assets_bucket and no way to configure a local path instead.
Hi!
Just set up the service but when trying to add an agency, it stops working, and this message appears on the server console:
Dec 14 19:40:19.190 [qtp2073480674-39] INFO spark.http.matching.MatcherFilter - The requested route [/api/editor/secure/agency] has not been mapped in Spark
Same happens with other functions like:
Dec 14 19:39:55.922 [qtp2073480674-16] INFO spark.http.matching.MatcherFilter - The requested route [/api/editor/secure/agency/new/uploadbranding] has not been mapped in Spark
Any known issues there?
Thanks!
These endpoints are currently using the TableWrite.update method to update the entity's branding field, which require a complete set of editor fields for an entity update. However, these branding URL values are generated by the server when the user uploads an image asset, so it should be safe to simply update the single field on the entity rather than using the complete Java API, which checks references, updates referring tables, etc.
on mongo-pojo branch, when I upload the attached GTFS file, a NPE is raised from r5 lib.
I am pretty sure it wasn't in previous version.
Anyway, whether or not some data are missing in the file , NPE should not occur.
11:02:56.935 [pool-2-thread-1] WARN c.c.d.manager.models.FeedVersion - Unknown error encountered while building network for RATP_GTFS_BUS_20-20171031T110251-01-81893661-6226-4a5d-b12b-17b2c22ca0af.zip. 11:02:56.940 [pool-2-thread-1] WARN c.c.d.m.jobs.ProcessSingleFeedJob - Not storing version RATP_GTFS_BUS_20-20171031T110251-01-81893661-6226-4a5d-b12b-17b2c22ca0af.zip because error occurred during processing. java.lang.NullPointerException at com.conveyal.r5.streets.StreetLayer.getVertexIndexForOsmNode(StreetLayer.java:883) at com.conveyal.r5.streets.StreetLayer.makeEdge(StreetLayer.java:927) at com.conveyal.r5.streets.StreetLayer.loadFromOsm(StreetLayer.java:264) at com.conveyal.r5.streets.StreetLayer.loadFromOsm(StreetLayer.java:170) at com.conveyal.r5.transit.TransportNetwork.fromFiles(TransportNetwork.java:166) at com.conveyal.r5.transit.TransportNetwork.fromFeeds(TransportNetwork.java:133) at com.conveyal.datatools.manager.models.FeedVersion.buildTransportNetwork(FeedVersion.java:365) at com.conveyal.datatools.manager.jobs.BuildTransportNetworkJob.jobLogic(BuildTransportNetworkJob.java:33) at com.conveyal.datatools.common.status.MonitorableJob.run(MonitorableJob.java:108) at com.conveyal.datatools.common.status.MonitorableJob.run(MonitorableJob.java:126) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
When deploying the datatools-server for the first time, and if you put different directories for each properties requiring a path on the disk filesystem you got the error below when importing a new feedversion. This is due to a missing directory not created by the datatools on first hand.
Fix proposal : check that the directory exists and create it otherwise.
14:48:31.250 [qtp1641596316-19] WARN org.eclipse.jetty.server.HttpChannel - //localhost:4200/api/manager/secure/feedversion?feedSourceId=07b8812b-bce6-40b1-b72a-f3324bbe837a&lastModified=1503075401000 java.lang.ExceptionInInitializerError: null at com.conveyal.datatools.manager.controllers.api.FeedVersionController.createFeedVersion(FeedVersionController.java:112) ~[classes/:na] at spark.ResponseTransformerRouteImpl$1.handle(ResponseTransformerRouteImpl.java:47) ~[spark-core-2.5.jar:na] at spark.http.matching.Routes.execute(Routes.java:61) ~[spark-core-2.5.jar:na] at spark.http.matching.MatcherFilter.doFilter(MatcherFilter.java:126) ~[spark-core-2.5.jar:na] at spark.embeddedserver.jetty.JettyHandler.doHandle(JettyHandler.java:50) ~[spark-core-2.5.jar:na] at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:189) ~[jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119) ~[jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.server.Server.handle(Server.java:517) ~[jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) ~[jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:242) [jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261) [jetty-io-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) [jetty-io-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:75) [jetty-io-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:213) [jetty-util-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147) [jetty-util-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) [jetty-util-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) [jetty-util-9.3.6.v20151106.jar:9.3.6.v20151106] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] Caused by: java.lang.IllegalArgumentException: Not a directory or not found: D:\tmp\data\gtfs at com.conveyal.datatools.manager.persistence.FeedStore.getPath(FeedStore.java:95) ~[classes/:na] at com.conveyal.datatools.manager.persistence.FeedStore.<init>(FeedStore.java:66) ~[classes/:na] at com.conveyal.datatools.manager.persistence.FeedStore.<init>(FeedStore.java:54) ~[classes/:na] at com.conveyal.datatools.manager.models.FeedVersion.<clinit>(FeedVersion.java:81) ~[classes/:na] ... 19 common frames omitted
In the network monitor tab of my browser, I see a 404 recorded to the ssodata-endpoint of Auth0. This may caused by the ssodata endpoint deprecation. According to a discussion, new tenants don’t have the ability to use older versions of Lock in embedded form and they also don’t have access to old legacy grant flows.
Does this mean, there is currently no means to successfully install and use catalogueglobal/datatools-server if you do not have an old Auth0 tenant?
Is it possible to deploy this via Docker?
Received this error when performing an update:
Unterminated string literal started at position 109 in SQL select id from mzwm_hhhdiiqoismphsklchhmrx.feed_info where feed_publisher_name = '' end-to-end automated test'. Expected char
Following all instructions on https://conveyal-data-tools.readthedocs.io/en/latest/dev/deployment/ I get the following warnings:
log4j:WARN No appenders could be found for logger (org.apache.http.client.protocol.RequestAddCookies).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
http://logging.apache.org/log4j/1.2/faq.html#noconfig basically tells me I should throw a log4j.properties or log4j.xml somewhere into the class path. It would be nice to have a section about that in the docs, though to allow me understand what the application does (which is kind of hard without any logs).
Running into packaging issues because the tests use env.yml.tmp instead of env.yml. I assume it should use that at least as server.yml is also being used. When env.tmp.yml is used then errors are encountered as GTFS_DATABASE_USER and GTFS_DATABASE_PASSWORD are both commented out and empty.
Hi,
When I try to build the project I am getting this error when the tests start: http://prntscr.com/hj0j3v
I do not see any info about a mongodb setup in the docs. I've started up mongodb locally but nothing changed. Any suggestions?
...
09:10:36.239 [Finalizer] ERROR c.c.d.editor.datastore.DatabaseTx - DB transaction left unclosed, this signifies a memory leak!
...
...
13:21:29.491 [Thread-11] INFO com.conveyal.gtfs.GTFSFeed - DuplicateStopsValidator finished in 2092 milliseconds.
13:21:29.492 [Thread-11] INFO com.conveyal.gtfs.GTFSFeed - HopSpeedsReasonableValidator finished in 0 milliseconds.
13:21:29.630 [Thread-11] INFO com.conveyal.gtfs.GTFSFeed - MisplacedStopValidator finished in 137 milliseconds.
13:21:29.632 [Thread-11] INFO com.conveyal.gtfs.GTFSFeed - NamesValidator finished in 1 milliseconds.
13:21:32.257 [Thread-11] INFO com.conveyal.gtfs.GTFSFeed - OverlappingTripsValidator finished in 2625 milliseconds.
13:21:33.109 [Thread-11] ERROR com.conveyal.gtfs.GTFSFeed - Could not run ReversedTripsValidator validator.
java.lang.NullPointerException
at com.conveyal.gtfs.validator.ReversedTripsValidator.validate(ReversedTripsValidator.java:68)
at com.conveyal.gtfs.GTFSFeed.validate(GTFSFeed.java:284)
at com.conveyal.gtfs.GTFSFeed.validate(GTFSFeed.java:301)
at com.conveyal.datatools.manager.models.FeedVersion.validate(FeedVersion.java:282)
at com.conveyal.datatools.manager.jobs.ValidateFeedJob.run(ValidateFeedJob.java:45)
at java.lang.Thread.run(Thread.java:745)
13:21:35.756 [Thread-11] INFO com.conveyal.gtfs.GTFSFeed - TripTimesValidator finished in 2643 milliseconds.
13:21:35.761 [Thread-11] INFO com.conveyal.gtfs.GTFSFeed - UnusedStopValidator finished in 4 milliseconds.
13:21:35.761 [Thread-11] INFO com.conveyal.gtfs.GTFSFeed - 8 validators completed in 8362 milliseconds.
13:21:35.761 [Thread-11] INFO c.c.d.manager.models.FeedVersion - Calculating stats...
13:21:37.948 [Thread-11] INFO c.c.d.manager.models.FeedVersion - Calculating # of trips per date of service
13:21:51.062 [Thread-11] INFO c.c.d.manager.models.FeedVersion - Total errors after validation: 9
13:21:51.063 [Thread-11] INFO c.c.d.manager.models.FeedVersion - Uploading validation json to S3
...
Problem
There's no explicit contribution process, making it harder for contributors to propose fixes, and making the overall contributing experience rather confusing.
See /pull/68
Possible solution
It would be superb if could explain the expected contribution mechanism in a CONTRIBUTING.md file at the root of this repository.
See for example https://github.com/openfisca/openfisca-core/blob/master/CONTRIBUTING.md
On mongo-pojo branch at commit number 7b1b4d8e30c81b8ffe0d6dfeaaedc4f3708b5e77:
The processes of creating a project, creating a feed source and creating a feed version through an upload of GTFS zipped file have all succeeded. The two databases, postgresql and mongo, have now the corresponding data. Then in the Version summary of the UI I see 0 Agency Count, 0 Route Count and 0 Tip Count. This is not true! None of the corresponding table is empty.
Then when I switch to any of the tabs Routes, Patterns or Stops, nothing happens on the UI except the "waiting animation".
The javascript console displays the following:
routes.js:34 GET http://localhost:4000/api/manager/graphql?query=%0Aquery%20routeQuery(%24feedId%3A%20%5BString%5D)%20%7B%0A%20%20routes(feed_id%3A%20%24feedId)%20%7B%0A%20%20%20%20route_id%0A%20%20%20%20route_short_name%0A%20%20%20%20route_long_name%2C%0A%20%20%20%20route_desc%2C%0A%20%20%20%20route_url%2C%0A%20%20%20%20trip_count%2C%0A%20%20%20%20pattern_count%0A%20%20%7D%0A%7D%0A&variables=%7B%22feedId%22%3A%22TISSEO-20170914T142615-02-bc15c0a0-d47e-4b20-8d51-87537956c281%22%7D 400 (Bad Request)
(anonymous) @ routes.js:34
(anonymous) @ index.js:11
(anonymous) @ promise.js:16
(anonymous) @ multi.js:12
(anonymous) @ fetch.js:70
(anonymous) @ middleware.js:22
onComponentMount @ Routes.js:18
value @ RouteLayout.js:15
performInitialMount @ ReactCompositeComponent.js:351
mountComponent @ ReactCompositeComponent.js:258
mountComponent @ ReactReconciler.js:46
performInitialMount @ ReactCompositeComponent.js:371
mountComponent @ ReactCompositeComponent.js:258
mountComponent @ ReactReconciler.js:46
mountChildren @ ReactMultiChild.js:238
_createInitialChildren @ ReactDOMComponent.js:697
mountComponent @ ReactDOMComponent.js:516
mountComponent @ ReactReconciler.js:46
_updateRenderedComponent @ ReactCompositeComponent.js:765
_performComponentUpdate @ ReactCompositeComponent.js:724
updateComponent @ ReactCompositeComponent.js:645
receiveComponent @ ReactCompositeComponent.js:547
receiveComponent @ ReactReconciler.js:125
_updateRenderedComponent @ ReactCompositeComponent.js:754
_performComponentUpdate @ ReactCompositeComponent.js:724
updateComponent @ ReactCompositeComponent.js:645
receiveComponent @ ReactCompositeComponent.js:547
receiveComponent @ ReactReconciler.js:125
_updateRenderedComponent @ ReactCompositeComponent.js:754
_performComponentUpdate @ ReactCompositeComponent.js:724
updateComponent @ ReactCompositeComponent.js:645
receiveComponent @ ReactCompositeComponent.js:547
receiveComponent @ ReactReconciler.js:125
_updateRenderedComponent @ ReactCompositeComponent.js:754
_performComponentUpdate @ ReactCompositeComponent.js:724
updateComponent @ ReactCompositeComponent.js:645
receiveComponent @ ReactCompositeComponent.js:547
receiveComponent @ ReactReconciler.js:125
updateChildren @ ReactChildReconciler.js:109
_reconcilerUpdateChildren @ ReactMultiChild.js:213
_updateChildren @ ReactMultiChild.js:312
updateChildren @ ReactMultiChild.js:299
_updateDOMChildren @ ReactDOMComponent.js:936
updateComponent @ ReactDOMComponent.js:754
receiveComponent @ ReactDOMComponent.js:716
receiveComponent @ ReactReconciler.js:125
_updateRenderedComponent @ ReactCompositeComponent.js:754
_performComponentUpdate @ ReactCompositeComponent.js:724
updateComponent @ ReactCompositeComponent.js:645
performUpdateIfNecessary @ ReactCompositeComponent.js:561
performUpdateIfNecessary @ ReactReconciler.js:157
runBatchedUpdates @ ReactUpdates.js:150
perform @ Transaction.js:140
perform @ Transaction.js:140
perform @ ReactUpdates.js:89
flushBatchedUpdates @ ReactUpdates.js:172
closeAll @ Transaction.js:206
perform @ Transaction.js:153
batchedUpdates @ ReactDefaultBatchingStrategy.js:62
enqueueUpdate @ ReactUpdates.js:200
enqueueUpdate @ ReactUpdateQueue.js:24
enqueueCallback @ ReactUpdateQueue.js:123
ReactComponent.setState @ ReactComponent.js:65
value @ Transition.js:181
(anonymous) @ Transition.js:155
nextCallback @ Transition.js:195
routes.js:39 Uncaught (in promise) TypeError: Cannot read property 'length' of undefined
at reducer (routes.js:39)
at combineReducers.js:132
at combineReducers.js:132
at c (createStore.js:179)
at index.js:14
at promise.js:16
at multi.js:12
at fetch.js:70
at middleware.js:22
at dispatch (applyMiddleware.js:45)
and in the datatools-server logs displays the following:
sept. 14 14:30:10.511 [qtp355582964-19] INFO graphql.GraphQL - Executing request. operation name: null. Request:
query routeQuery($feedId: [String]) {
routes(feed_id: $feedId) {
route_id
route_short_name
route_long_name,
route_desc,
route_url,
trip_count,
pattern_count
}
}
When running mvn package, env.yml.tmp and server.yml.tmp are consumed instead of those without .tpl extension. This is undocumented and unexpted. Tests fail if no mongodb is running on localhost and on the other hand, if a mongodb is running at localhost it is accessed during mvn package, which might be unwanted or unexpected.
The official deployment manual says:
Copy the included configuration env templates for both the server and UI repos
and later
Package the application using Maven
However, it fails to mention that hard coded test.
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.conveyal.datatools.editor.ProcessGtfsSnapshotMergeTest
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/zbouziane/.m2/repository/ch/qos/logback/logback-classic/1.1.3/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/zbouziane/.m2/repository/org/slf4j/slf4j-simple/1.7.10/slf4j-simple-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
17:12:57,786 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
17:12:57,787 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
17:12:57,787 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [file:/Users/zbouziane/tmp/tmp/datatools-server/target/classes/logback.xml]
17:12:57,788 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs multiple times on the classpath.
17:12:57,788 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [jar:file:/Users/zbouziane/.m2/repository/com/conveyal/osm-lib/1.1.0-SNAPSHOT/osm-lib-1.1.0-SNAPSHOT.jar!/logback.xml]
17:12:57,788 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [file:/Users/zbouziane/tmp/tmp/datatools-server/target/classes/logback.xml]
17:12:57,973 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
17:13:02,984 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
17:13:02,991 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
17:13:03,017 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
17:13:03,069 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to DEBUG
17:13:03,069 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
17:13:03,070 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
17:13:03,071 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@22eeefeb - Registering current configuration as safe fallback point
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
17/04/19 17:13:03.080 [main] INFO com.conveyal.datatools.DatatoolsTest - DatatoolsTest setup
17/04/19 17:13:03.272 [main] INFO c.c.datatools.manager.DataManager - Loading env.yml: configurations/default/env.yml.tmp
17/04/19 17:13:03.274 [main] INFO c.c.datatools.manager.DataManager - Loading server.yml: configurations/default/server.yml
17/04/19 17:13:03.400 [main] INFO c.c.d.manager.persistence.DataStore - /private/tmp/projects
17/04/19 17:13:03.574 [main] WARN c.c.datatools.manager.DataManager - Config property application.data.gtfs_s3_bucket not found
17/04/19 17:13:03.575 [main] WARN c.c.datatools.manager.DataManager - Config property application.data.gtfs_s3_bucket not found
17/04/19 17:13:03.584 [main] WARN c.c.datatools.manager.DataManager - Config property application.data.s3_credentials_file not found
17/04/19 17:13:03.584 [main] WARN c.c.datatools.manager.DataManager - Config property application.data.s3_credentials_file not found
17/04/19 17:13:03.584 [main] INFO c.c.datatools.manager.DataManager - Initializing gtfs-api cache locally (no s3 bucket) /tmp
17/04/19 17:13:04.450 [main] INFO com.conveyal.gtfs.BaseGTFSCache - No bucket specified; GTFS Cache will run locally
17/04/19 17:13:04.489 [main] DEBUG spark.route.Routes - Adds route: after, +/*paths, spark.FilterImpl$1@30af5b6b
17/04/19 17:13:04.522 [Thread-1] DEBUG org.eclipse.jetty.util.log - Logging to Logger[org.eclipse.jetty.util.log] via org.eclipse.jetty.util.log.Slf4jLog
17/04/19 17:13:04.528 [Thread-1] INFO org.eclipse.jetty.util.log - Logging initialized @7553ms
17/04/19 17:13:04.539 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - spark.embeddedserver.jetty.JettyHandler@740097e2 added {org.eclipse.jetty.server.session.HashSessionManager@9af482e,AUTO}
17/04/19 17:13:04.559 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/project/:id, spark.ResponseTransformerRouteImpl$1@2c1156a7
17/04/19 17:13:04.559 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - org.eclipse.jetty.server.Server@2e01be57 added {qtp1978280845{STOPPED,8<=0<=200,i=0,q=0},AUTO}
17/04/19 17:13:04.561 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/project, spark.ResponseTransformerRouteImpl$1@4f1bfe23
17/04/19 17:13:04.563 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/project, spark.ResponseTransformerRouteImpl$1@71529963
17/04/19 17:13:04.568 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/project/:id, spark.ResponseTransformerRouteImpl$1@5adb0db3
17/04/19 17:13:04.625 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/project/:id, spark.ResponseTransformerRouteImpl$1@7241a47d
17/04/19 17:13:04.626 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/project/:id/thirdPartySync/:type, spark.ResponseTransformerRouteImpl$1@5dcd8c7a
17/04/19 17:13:04.628 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/project/:id/fetch, spark.ResponseTransformerRouteImpl$1@253d9f73
17/04/19 17:13:04.630 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/project/:id/deployPublic, spark.ResponseTransformerRouteImpl$1@8dbfffb
17/04/19 17:13:04.631 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/project/:id/download, spark.RouteImpl$1@a22cb6a
17/04/19 17:13:04.632 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/project/:id, spark.ResponseTransformerRouteImpl$1@2e2ff723
17/04/19 17:13:04.634 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/project, spark.ResponseTransformerRouteImpl$1@2445445a
17/04/19 17:13:04.644 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/feedsource/:id, spark.ResponseTransformerRouteImpl$1@63798ca7
17/04/19 17:13:04.645 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/feedsource, spark.ResponseTransformerRouteImpl$1@5674e1f2
17/04/19 17:13:04.646 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/feedsource, spark.ResponseTransformerRouteImpl$1@32f232a5
17/04/19 17:13:04.648 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - HttpConnectionFactory@1f00700c[HTTP/1.1] added {HttpConfiguration@5520a96e{32768/8192,8192/8192,https://:0,[]},POJO}
17/04/19 17:13:04.649 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/feedsource/:id, spark.ResponseTransformerRouteImpl$1@e8df99a
17/04/19 17:13:04.650 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/feedsource/:id/updateExternal, spark.ResponseTransformerRouteImpl$1@517566b
17/04/19 17:13:04.651 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - ServerConnector@1a06506c{null,[]}{0.0.0.0:0} added {org.eclipse.jetty.server.Server@2e01be57,UNMANAGED}
17/04/19 17:13:04.652 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/feedsource/:id, spark.ResponseTransformerRouteImpl$1@2df6226d
17/04/19 17:13:04.652 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - ServerConnector@1a06506c{null,[]}{0.0.0.0:0} added {qtp1978280845{STOPPED,8<=0<=200,i=0,q=0},AUTO}
17/04/19 17:13:04.653 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - ServerConnector@1a06506c{null,[]}{0.0.0.0:0} added {org.eclipse.jetty.util.thread.ScheduledExecutorScheduler@112a1b5d,AUTO}
17/04/19 17:13:04.653 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - ServerConnector@1a06506c{null,[]}{0.0.0.0:0} added {org.eclipse.jetty.io.ArrayByteBufferPool@2a5887b5,POJO}
17/04/19 17:13:04.654 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - ServerConnector@1a06506c{null,[http/1.1]}{0.0.0.0:0} added {HttpConnectionFactory@1f00700c[HTTP/1.1],AUTO}
17/04/19 17:13:04.654 [Thread-1] DEBUG o.e.jetty.server.ServerConnector - ServerConnector@1a06506c{HTTP/1.1,[http/1.1]}{0.0.0.0:0} added HttpConnectionFactory@1f00700c[HTTP/1.1]
17/04/19 17:13:04.657 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - ServerConnector@1a06506c{HTTP/1.1,[http/1.1]}{0.0.0.0:0} added {org.eclipse.jetty.server.ServerConnector$ServerConnectorManager@31efc5b5,MANAGED}
17/04/19 17:13:04.658 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - org.eclipse.jetty.server.Server@2e01be57 added {ServerConnector@1a06506c{HTTP/1.1,[http/1.1]}{0.0.0.0:4000},AUTO}
17/04/19 17:13:04.662 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - org.eclipse.jetty.server.Server@2e01be57 added {spark.embeddedserver.jetty.JettyHandler@740097e2,MANAGED}
17/04/19 17:13:04.662 [Thread-1] INFO s.e.jetty.EmbeddedJettyServer - == Spark has ignited ...
17/04/19 17:13:04.662 [Thread-1] INFO s.e.jetty.EmbeddedJettyServer - >> Listening on 0.0.0.0:4000
17/04/19 17:13:04.663 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting org.eclipse.jetty.server.Server@2e01be57
17/04/19 17:13:04.667 [Thread-1] INFO org.eclipse.jetty.server.Server - jetty-9.3.6.v20151106
17/04/19 17:13:04.701 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/feedsource/:id/fetch, spark.ResponseTransformerRouteImpl$1@55322aab
17/04/19 17:13:04.703 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/feedsource/:id, spark.ResponseTransformerRouteImpl$1@50468873
17/04/19 17:13:04.705 [Thread-1] DEBUG o.e.j.server.handler.AbstractHandler - starting org.eclipse.jetty.server.Server@2e01be57
17/04/19 17:13:04.705 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/feedsource, spark.ResponseTransformerRouteImpl$1@76884e4b
17/04/19 17:13:04.705 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting qtp1978280845{STOPPED,8<=0<=200,i=0,q=0}
17/04/19 17:13:04.707 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7732ms qtp1978280845{STARTED,8<=8<=200,i=7,q=0}
17/04/19 17:13:04.707 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting spark.embeddedserver.jetty.JettyHandler@740097e2
17/04/19 17:13:04.707 [Thread-1] DEBUG o.e.j.server.handler.AbstractHandler - starting spark.embeddedserver.jetty.JettyHandler@740097e2
17/04/19 17:13:04.707 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting org.eclipse.jetty.server.session.HashSessionManager@9af482e
17/04/19 17:13:04.712 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - org.eclipse.jetty.server.session.HashSessionManager@9af482e added {org.eclipse.jetty.util.thread.ScheduledExecutorScheduler@1c1018ce,MANAGED}
17/04/19 17:13:04.715 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting org.eclipse.jetty.server.session.HashSessionIdManager@1fa25d56
17/04/19 17:13:04.715 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7741ms org.eclipse.jetty.server.session.HashSessionIdManager@1fa25d56
17/04/19 17:13:04.716 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - org.eclipse.jetty.server.Server@2e01be57 added {org.eclipse.jetty.server.session.HashSessionIdManager@1fa25d56,MANAGED}
17/04/19 17:13:04.716 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - org.eclipse.jetty.server.session.HashSessionManager@9af482e added {org.eclipse.jetty.server.session.HashSessionIdManager@1fa25d56,UNMANAGED}
17/04/19 17:13:04.716 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting org.eclipse.jetty.util.thread.ScheduledExecutorScheduler@1c1018ce
17/04/19 17:13:04.716 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/feedversion/:id, spark.ResponseTransformerRouteImpl$1@62727399
17/04/19 17:13:04.717 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7742ms org.eclipse.jetty.util.thread.ScheduledExecutorScheduler@1c1018ce
17/04/19 17:13:04.717 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/feedversion/:id/download, spark.RouteImpl$1@130dca52
17/04/19 17:13:04.720 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7745ms org.eclipse.jetty.server.session.HashSessionManager@9af482e
17/04/19 17:13:04.720 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7746ms spark.embeddedserver.jetty.JettyHandler@740097e2
17/04/19 17:13:04.721 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting ServerConnector@1a06506c{HTTP/1.1,[http/1.1]}{0.0.0.0:4000}
17/04/19 17:13:04.728 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/feedversion/:id/downloadtoken, spark.ResponseTransformerRouteImpl$1@15dcfae7
17/04/19 17:13:04.732 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/feedversion/:id/validation, spark.ResponseTransformerRouteImpl$1@115667d
17/04/19 17:13:04.734 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/feedversion/:id/validate, spark.ResponseTransformerRouteImpl$1@497486b3
17/04/19 17:13:04.744 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/feedversion/:id/isochrones, spark.ResponseTransformerRouteImpl$1@6127a7e
17/04/19 17:13:04.746 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/feedversion, spark.ResponseTransformerRouteImpl$1@76012793
17/04/19 17:13:04.748 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/feedversion, spark.ResponseTransformerRouteImpl$1@4dd02341
17/04/19 17:13:04.749 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/feedversion/fromsnapshot, spark.ResponseTransformerRouteImpl$1@495b0487
17/04/19 17:13:04.750 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - ServerConnector@1a06506c{HTTP/1.1,[http/1.1]}{0.0.0.0:4000} added {sun.nio.ch.ServerSocketChannelImpl[/0:0:0:0:0:0:0:0:4000],POJO}
17/04/19 17:13:04.751 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting org.eclipse.jetty.util.thread.ScheduledExecutorScheduler@112a1b5d
17/04/19 17:13:04.751 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7777ms org.eclipse.jetty.util.thread.ScheduledExecutorScheduler@112a1b5d
17/04/19 17:13:04.752 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting HttpConnectionFactory@1f00700c[HTTP/1.1]
17/04/19 17:13:04.753 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7777ms HttpConnectionFactory@1f00700c[HTTP/1.1]
17/04/19 17:13:04.753 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting org.eclipse.jetty.server.ServerConnector$ServerConnectorManager@31efc5b5
17/04/19 17:13:04.762 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/feedversion/:id/rename, spark.ResponseTransformerRouteImpl$1@93cf163
17/04/19 17:13:04.764 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/feedversion/:id/publish, spark.ResponseTransformerRouteImpl$1@25bfcafd
17/04/19 17:13:04.767 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/feedversion/:id, spark.ResponseTransformerRouteImpl$1@2a76840c
17/04/19 17:13:04.768 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/feedversion, spark.ResponseTransformerRouteImpl$1@7cf6a5f9
17/04/19 17:13:04.769 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting org.eclipse.jetty.io.ManagedSelector@5c0b94f8 id=0 keys=-1 selected=-1
17/04/19 17:13:04.769 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/feedversion/:id/validation, spark.ResponseTransformerRouteImpl$1@66565121
17/04/19 17:13:04.770 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/feedversion/:id/downloadtoken, spark.ResponseTransformerRouteImpl$1@6179e425
17/04/19 17:13:04.771 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/downloadfeed/:token, spark.RouteImpl$1@2d6764b2
17/04/19 17:13:04.776 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/region/:id, spark.ResponseTransformerRouteImpl$1@6399551e
17/04/19 17:13:04.779 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/region, spark.ResponseTransformerRouteImpl$1@5d5f10b2
17/04/19 17:13:04.781 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/region, spark.ResponseTransformerRouteImpl$1@53bd8fca
17/04/19 17:13:04.783 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7806ms org.eclipse.jetty.io.ManagedSelector@5c0b94f8 id=0 keys=0 selected=0
17/04/19 17:13:04.783 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/region/:id, spark.ResponseTransformerRouteImpl$1@5cde6747
17/04/19 17:13:04.784 [Thread-1] DEBUG o.e.j.util.thread.QueuedThreadPool - queue org.eclipse.jetty.io.ManagedSelector@5c0b94f8 id=0 keys=0 selected=0
17/04/19 17:13:04.786 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/region/:id, spark.ResponseTransformerRouteImpl$1@3ed242a4
17/04/19 17:13:04.787 [qtp1978280845-14] DEBUG o.e.j.util.thread.QueuedThreadPool - run org.eclipse.jetty.io.ManagedSelector@5c0b94f8 id=0 keys=0 selected=0
17/04/19 17:13:04.787 [qtp1978280845-14] DEBUG o.e.j.u.t.s.ExecuteProduceConsume - EPR Idle/org.eclipse.jetty.io.ManagedSelector$SelectorProducer@162fba2a execute
17/04/19 17:13:04.788 [qtp1978280845-14] DEBUG o.e.j.u.t.s.ExecuteProduceConsume - EPR Prod/org.eclipse.jetty.io.ManagedSelector$SelectorProducer@162fba2a produce enter
17/04/19 17:13:04.788 [qtp1978280845-14] DEBUG o.e.j.u.t.s.ExecuteProduceConsume - EPR Prod/org.eclipse.jetty.io.ManagedSelector$SelectorProducer@162fba2a producing
17/04/19 17:13:04.788 [qtp1978280845-14] DEBUG org.eclipse.jetty.io.ManagedSelector - Selector loop waiting on select
17/04/19 17:13:04.791 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/region/:id, spark.ResponseTransformerRouteImpl$1@1fdf1c5
17/04/19 17:13:04.793 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting org.eclipse.jetty.io.ManagedSelector@31b31a76 id=1 keys=-1 selected=-1
17/04/19 17:13:04.793 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/region, spark.ResponseTransformerRouteImpl$1@7d64e326
17/04/19 17:13:04.794 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7819ms org.eclipse.jetty.io.ManagedSelector@31b31a76 id=1 keys=0 selected=0
17/04/19 17:13:04.795 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/seedregions, spark.ResponseTransformerRouteImpl$1@68be8808
17/04/19 17:13:04.796 [Thread-1] DEBUG o.e.j.util.thread.QueuedThreadPool - queue org.eclipse.jetty.io.ManagedSelector@31b31a76 id=1 keys=0 selected=0
17/04/19 17:13:04.799 [qtp1978280845-15] DEBUG o.e.j.util.thread.QueuedThreadPool - run org.eclipse.jetty.io.ManagedSelector@31b31a76 id=1 keys=0 selected=0
17/04/19 17:13:04.799 [qtp1978280845-15] DEBUG o.e.j.u.t.s.ExecuteProduceConsume - EPR Idle/org.eclipse.jetty.io.ManagedSelector$SelectorProducer@1677891 execute
17/04/19 17:13:04.799 [qtp1978280845-15] DEBUG o.e.j.u.t.s.ExecuteProduceConsume - EPR Prod/org.eclipse.jetty.io.ManagedSelector$SelectorProducer@1677891 produce enter
17/04/19 17:13:04.799 [qtp1978280845-15] DEBUG o.e.j.u.t.s.ExecuteProduceConsume - EPR Prod/org.eclipse.jetty.io.ManagedSelector$SelectorProducer@1677891 producing
17/04/19 17:13:04.800 [qtp1978280845-15] DEBUG org.eclipse.jetty.io.ManagedSelector - Selector loop waiting on select
17/04/19 17:13:04.802 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7827ms org.eclipse.jetty.server.ServerConnector$ServerConnectorManager@31efc5b5
17/04/19 17:13:04.804 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/note, spark.ResponseTransformerRouteImpl$1@ebaa6cb
17/04/19 17:13:04.807 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/note, spark.ResponseTransformerRouteImpl$1@5fe1ce85
17/04/19 17:13:04.809 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - ServerConnector@1a06506c{HTTP/1.1,[http/1.1]}{0.0.0.0:4000} added {acceptor-0@4c5a3f8a,POJO}
17/04/19 17:13:04.811 [Thread-1] DEBUG o.e.j.util.thread.QueuedThreadPool - queue acceptor-0@4c5a3f8a
17/04/19 17:13:04.812 [qtp1978280845-16] DEBUG o.e.j.util.thread.QueuedThreadPool - run acceptor-0@4c5a3f8a
17/04/19 17:13:04.813 [Thread-1] INFO o.e.jetty.server.ServerConnector - Started ServerConnector@1a06506c{HTTP/1.1,[http/1.1]}{0.0.0.0:4000}
17/04/19 17:13:04.815 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7840ms ServerConnector@1a06506c{HTTP/1.1,[http/1.1]}{0.0.0.0:4000}
17/04/19 17:13:04.817 [Thread-1] INFO org.eclipse.jetty.server.Server - Started @7841ms
17/04/19 17:13:04.817 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7842ms org.eclipse.jetty.server.Server@2e01be57
17/04/19 17:13:04.817 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/status/jobs, spark.ResponseTransformerRouteImpl$1@2a640157
17/04/19 17:13:04.821 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/organization/:id, spark.ResponseTransformerRouteImpl$1@f80945f
17/04/19 17:13:04.822 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/organization, spark.ResponseTransformerRouteImpl$1@5ef6ae06
17/04/19 17:13:04.823 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/organization, spark.ResponseTransformerRouteImpl$1@1cdc4c27
17/04/19 17:13:04.824 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/organization/:id, spark.ResponseTransformerRouteImpl$1@1e8ce150
17/04/19 17:13:04.825 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/organization/:id, spark.ResponseTransformerRouteImpl$1@1b73be9f
17/04/19 17:13:04.902 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/agency/:id, spark.ResponseTransformerRouteImpl$1@232a7d73
17/04/19 17:13:04.903 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/agency, spark.RouteImpl$1@22ffa91a
17/04/19 17:13:04.904 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/agency, spark.ResponseTransformerRouteImpl$1@40844aab
17/04/19 17:13:04.906 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/agency, spark.ResponseTransformerRouteImpl$1@66746f57
17/04/19 17:13:04.908 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/agency/:id, spark.ResponseTransformerRouteImpl$1@28e8dde3
17/04/19 17:13:04.909 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/agency/:id/duplicate, spark.ResponseTransformerRouteImpl$1@1817f1eb
17/04/19 17:13:04.910 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/agency/:id/uploadbranding, spark.ResponseTransformerRouteImpl$1@301ec38b
17/04/19 17:13:04.911 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/agency/:id, spark.ResponseTransformerRouteImpl$1@c5dc4a2
17/04/19 17:13:04.915 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/calendar/:id, spark.ResponseTransformerRouteImpl$1@5669c5fb
17/04/19 17:13:04.915 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/calendar, spark.RouteImpl$1@5f9678e1
17/04/19 17:13:04.916 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/calendar, spark.ResponseTransformerRouteImpl$1@5a9d6f02
17/04/19 17:13:04.918 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/calendar, spark.ResponseTransformerRouteImpl$1@5ccbeb64
17/04/19 17:13:04.921 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/calendar/:id, spark.ResponseTransformerRouteImpl$1@3243b914
17/04/19 17:13:04.922 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/calendar/:id, spark.ResponseTransformerRouteImpl$1@626c44e7
17/04/19 17:13:04.926 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/route/:id, spark.ResponseTransformerRouteImpl$1@3016fd5e
17/04/19 17:13:04.927 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/route, spark.RouteImpl$1@53d102a2
17/04/19 17:13:04.928 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/route, spark.ResponseTransformerRouteImpl$1@5aac4250
17/04/19 17:13:04.929 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/route/merge, spark.ResponseTransformerRouteImpl$1@59f63e24
17/04/19 17:13:04.929 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/route, spark.ResponseTransformerRouteImpl$1@fade1fc
17/04/19 17:13:04.930 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/route/:id, spark.ResponseTransformerRouteImpl$1@613a8ee1
17/04/19 17:13:04.931 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/route/:id/uploadbranding, spark.ResponseTransformerRouteImpl$1@1b765a2c
17/04/19 17:13:04.932 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/route/:id, spark.ResponseTransformerRouteImpl$1@18920cc
17/04/19 17:13:04.935 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/routetype/:id, spark.ResponseTransformerRouteImpl$1@39a8312f
17/04/19 17:13:04.935 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/routetype, spark.RouteImpl$1@2c532cd8
17/04/19 17:13:04.936 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/routetype, spark.ResponseTransformerRouteImpl$1@3700ec9c
17/04/19 17:13:04.937 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/routetype, spark.ResponseTransformerRouteImpl$1@31000e60
17/04/19 17:13:04.939 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/routetype/:id, spark.ResponseTransformerRouteImpl$1@54c62d71
17/04/19 17:13:04.940 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/routetype/:id, spark.ResponseTransformerRouteImpl$1@2d1dee39
17/04/19 17:13:04.966 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/scheduleexception/:id, spark.ResponseTransformerRouteImpl$1@4af0df05
17/04/19 17:13:04.967 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/scheduleexception, spark.RouteImpl$1@acdb094
17/04/19 17:13:04.969 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/scheduleexception, spark.ResponseTransformerRouteImpl$1@2ceb80a1
17/04/19 17:13:04.971 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/scheduleexception, spark.ResponseTransformerRouteImpl$1@2072acb2
17/04/19 17:13:04.974 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/scheduleexception/:id, spark.ResponseTransformerRouteImpl$1@70a36a66
17/04/19 17:13:04.977 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/scheduleexception/:id, spark.ResponseTransformerRouteImpl$1@d3957fe
17/04/19 17:13:04.985 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/stop/:id, spark.ResponseTransformerRouteImpl$1@158a3b2e
17/04/19 17:13:04.985 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/stop, spark.RouteImpl$1@5e1fa5b1
17/04/19 17:13:04.986 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/stop, spark.ResponseTransformerRouteImpl$1@3e821657
17/04/19 17:13:04.987 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/stop/mergeStops, spark.ResponseTransformerRouteImpl$1@615091b8
17/04/19 17:13:04.988 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/stop, spark.ResponseTransformerRouteImpl$1@43b6123e
17/04/19 17:13:04.989 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/stop/:id, spark.ResponseTransformerRouteImpl$1@12bd8a64
17/04/19 17:13:04.990 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/stop/:id, spark.ResponseTransformerRouteImpl$1@ebb6851
17/04/19 17:13:04.997 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/trip/:id, spark.ResponseTransformerRouteImpl$1@3a6f2de3
17/04/19 17:13:04.997 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/trip, spark.RouteImpl$1@56303b57
17/04/19 17:13:04.999 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/trip, spark.ResponseTransformerRouteImpl$1@78f5c518
17/04/19 17:13:05.000 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/trip, spark.ResponseTransformerRouteImpl$1@753432a2
17/04/19 17:13:05.001 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/trip/:id, spark.ResponseTransformerRouteImpl$1@44e3a2b2
17/04/19 17:13:05.002 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/trip/:id, spark.ResponseTransformerRouteImpl$1@1d2bd371
17/04/19 17:13:05.015 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/trippattern/:id, spark.ResponseTransformerRouteImpl$1@26bab2f1
17/04/19 17:13:05.017 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/trippattern, spark.RouteImpl$1@68ead359
17/04/19 17:13:05.018 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/trippattern, spark.ResponseTransformerRouteImpl$1@a43ce46
17/04/19 17:13:05.019 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/trippattern, spark.ResponseTransformerRouteImpl$1@37052337
17/04/19 17:13:05.020 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/trippattern/:id, spark.ResponseTransformerRouteImpl$1@4d722ac9
17/04/19 17:13:05.021 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/trippattern/:id, spark.ResponseTransformerRouteImpl$1@3d97a632
17/04/19 17:13:05.030 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/snapshot/:id, spark.ResponseTransformerRouteImpl$1@60957c0f
17/04/19 17:13:05.031 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/snapshot, spark.RouteImpl$1@fcb4004
17/04/19 17:13:05.032 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/snapshot, spark.ResponseTransformerRouteImpl$1@2dd29a59
17/04/19 17:13:05.034 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/snapshot, spark.ResponseTransformerRouteImpl$1@40dff0b7
17/04/19 17:13:05.035 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/snapshot/import, spark.ResponseTransformerRouteImpl$1@1253e7cb
17/04/19 17:13:05.036 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/snapshot/:id, spark.ResponseTransformerRouteImpl$1@3b0c9195
17/04/19 17:13:05.037 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/snapshot/:id/restore, spark.ResponseTransformerRouteImpl$1@302a07d
17/04/19 17:13:05.042 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/snapshot/:id/downloadtoken, spark.ResponseTransformerRouteImpl$1@7e8dcdaa
17/04/19 17:13:05.045 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/snapshot/:id, spark.ResponseTransformerRouteImpl$1@5c08c46a
17/04/19 17:13:05.048 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/downloadsnapshot/:token, spark.RouteImpl$1@708400f6
17/04/19 17:13:05.055 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/feedinfo/:id, spark.ResponseTransformerRouteImpl$1@c35172e
17/04/19 17:13:05.056 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/feedinfo, spark.RouteImpl$1@3cc41abc
17/04/19 17:13:05.058 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/feedinfo, spark.ResponseTransformerRouteImpl$1@3668d4
17/04/19 17:13:05.059 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/feedinfo/:id, spark.ResponseTransformerRouteImpl$1@7f6f61c8
17/04/19 17:13:05.060 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/feedinfo/:id, spark.ResponseTransformerRouteImpl$1@4097cac
17/04/19 17:13:05.064 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/fare/:id, spark.ResponseTransformerRouteImpl$1@52d645b1
17/04/19 17:13:05.065 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/fare, spark.RouteImpl$1@2cc3ad05
17/04/19 17:13:05.068 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/fare, spark.ResponseTransformerRouteImpl$1@3d9f6567
17/04/19 17:13:05.069 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/fare, spark.ResponseTransformerRouteImpl$1@35e5d0e5
17/04/19 17:13:05.075 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/fare/:id, spark.ResponseTransformerRouteImpl$1@655ef322
17/04/19 17:13:05.076 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/fare/:id, spark.ResponseTransformerRouteImpl$1@5066d65f
17/04/19 17:13:05.101 [main] WARN c.c.d.m.c.api.GtfsApiController - No extension provided for GTFS API
17/04/19 17:13:05.101 [main] WARN c.c.datatools.manager.DataManager - Config property modules.gtfsapi.update_frequency not found
17/04/19 17:13:05.103 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//hello, spark.RouteImpl$1@20b12f8a
17/04/19 17:13:05.105 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//feeds, spark.ResponseTransformerRouteImpl$1@522a32b1
17/04/19 17:13:05.107 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//feeds/:id, spark.ResponseTransformerRouteImpl$1@5e01a982
17/04/19 17:13:05.110 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//routes, spark.ResponseTransformerRouteImpl$1@58fb7731
17/04/19 17:13:05.110 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//routes/:id, spark.ResponseTransformerRouteImpl$1@3fb6cf60
17/04/19 17:13:05.112 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//stops, spark.ResponseTransformerRouteImpl$1@5e8ac0e1
17/04/19 17:13:05.113 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//stops/:id, spark.ResponseTransformerRouteImpl$1@6955cb39
17/04/19 17:13:05.114 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//trips, spark.ResponseTransformerRouteImpl$1@6b7906b3
17/04/19 17:13:05.115 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//trips/:id, spark.ResponseTransformerRouteImpl$1@395b56bb
17/04/19 17:13:05.118 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//trips/:id/stoptimes, spark.ResponseTransformerRouteImpl$1@68fa0ba8
17/04/19 17:13:05.120 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//patterns, spark.ResponseTransformerRouteImpl$1@64a8c844
17/04/19 17:13:05.121 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//, spark.RouteImpl$1@52de51b6
17/04/19 17:13:05.127 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//graphql, spark.ResponseTransformerRouteImpl$1@2e9fda69
17/04/19 17:13:05.128 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager//graphql, spark.ResponseTransformerRouteImpl$1@1755e85b
17/04/19 17:13:05.130 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//graphql/schema, spark.ResponseTransformerRouteImpl$1@2371aaca
17/04/19 17:13:05.133 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager//graphql/schema, spark.ResponseTransformerRouteImpl$1@63fdab07
17/04/19 17:13:05.133 [main] WARN c.c.datatools.manager.DataManager - Config property modules.gtfsplus.enabled not found
17/04/19 17:13:05.133 [main] WARN c.c.datatools.manager.DataManager - Config property modules.gtfsplus.enabled not found
17/04/19 17:13:05.133 [main] WARN c.c.datatools.manager.DataManager - Config property modules.gtfsplus.enabled not found
17/04/19 17:13:05.147 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/user/:id, spark.ResponseTransformerRouteImpl$1@3f390d63
17/04/19 17:13:05.147 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/user/:id/recentactivity, spark.ResponseTransformerRouteImpl$1@2374d36a
17/04/19 17:13:05.148 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/user, spark.ResponseTransformerRouteImpl$1@47a5b70d
17/04/19 17:13:05.149 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/usercount, spark.ResponseTransformerRouteImpl$1@38f116f6
17/04/19 17:13:05.150 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/user, spark.ResponseTransformerRouteImpl$1@5c530d1e
17/04/19 17:13:05.152 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/user/:id, spark.ResponseTransformerRouteImpl$1@c5ee75e
17/04/19 17:13:05.153 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/user/:id, spark.ResponseTransformerRouteImpl$1@70efb718
17/04/19 17:13:05.157 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/public/user, spark.ResponseTransformerRouteImpl$1@4e858e0a
17/04/19 17:13:05.158 [main] WARN c.c.datatools.manager.DataManager - Config property modules.dump.enabled not found
17/04/19 17:13:05.159 [main] WARN c.c.datatools.manager.DataManager - Config property modules.dump.enabled not found
17/04/19 17:13:05.159 [main] WARN c.c.datatools.manager.DataManager - Config property modules.dump.enabled not found
17/04/19 17:13:05.161 [main] DEBUG spark.route.Routes - Adds route: before, /api/manager/secure/*, spark.FilterImpl$1@b7838a9
17/04/19 17:13:05.162 [main] DEBUG spark.route.Routes - Adds route: after, /api/manager/*, spark.FilterImpl$1@5733f295
17/04/19 17:13:05.163 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/*, spark.RouteImpl$1@47a86fbb
17/04/19 17:13:05.163 [main] DEBUG spark.route.Routes - Adds route: get, /*, spark.RouteImpl$1@19553973
17/04/19 17:13:05.164 [main] WARN c.c.datatools.manager.DataManager - Config property extensions.mtc.enabled not found
17/04/19 17:13:05.164 [main] WARN c.c.datatools.manager.DataManager - Config property extensions.mtc.enabled not found
17/04/19 17:13:05.164 [main] WARN c.c.datatools.manager.DataManager - Config property extensions.mtc.enabled not found
17/04/19 17:13:05.164 [main] INFO c.c.datatools.manager.DataManager - Registering TransitLand Resource
17/04/19 17:13:05.165 [main] INFO c.c.datatools.manager.DataManager - Registering TransitFeeds Resource
17/04/19 17:13:05.167 [main] INFO com.conveyal.datatools.LoadFeedTest - ProcessGtfsSnapshotMergeTest setup
17/04/19 17:13:05.169 [main] INFO c.c.d.manager.persistence.DataStore - /private/tmp/feedsources
17/04/19 17:13:05.295 [main] INFO c.c.d.manager.persistence.DataStore - /private/tmp/feedversions
17/04/19 17:13:05.389 [main] INFO c.c.d.manager.persistence.FeedStore - Writing file to /tmp/test-20170419T171305+02-f19c9daa-3bf3-4299-92a3-559a095fa45b.zip
17/04/19 17:13:05.403 [main] INFO c.c.d.manager.persistence.FeedStore - Copying version to latest <FeedSource test (f19c9daa-3bf3-4299-92a3-559a095fa45b)>
17/04/19 17:13:05.426 [main] INFO c.c.d.manager.models.FeedVersion - New GTFS file saved: test-20170419T171305+02-f19c9daa-3bf3-4299-92a3-559a095fa45b.zip
17/04/19 17:13:05.447 [main] INFO c.c.d.e.ProcessGtfsSnapshotMergeTest - ProcessGtfsSnapshotMergeTest setup
17/04/19 17:13:05.655 [main] INFO c.c.d.e.j.ProcessGtfsSnapshotMerge - GTFS Snapshot Merge for feedVersion test-20170419T171305+02-f19c9daa-3bf3-4299-92a3-559a095fa45b.zip
17/04/19 17:13:06.133 [main] INFO c.c.d.e.j.ProcessGtfsSnapshotMerge - Cleared old data
17/04/19 17:13:06.134 [main] WARN com.conveyal.gtfs.BaseGTFSCache - Feed test-20170419T171305+02-f19c9daa-3bf3-4299-92a3-559a095fa45b not found locally
java.lang.NullPointerException
at com.conveyal.datatools.manager.models.FeedVersion.getGtfsFeed(FeedVersion.java:182)
at com.conveyal.datatools.editor.jobs.ProcessGtfsSnapshotMerge.run(ProcessGtfsSnapshotMerge.java:130)
at com.conveyal.datatools.editor.ProcessGtfsSnapshotMergeTest.setUp(ProcessGtfsSnapshotMergeTest.java:39)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Tests run: 4, Failures: 4, Errors: 0, Skipped: 0, Time elapsed: 9.015 sec <<< FAILURE!
At the page :
http://conveyal-data-tools.readthedocs.io/en/dev/dev/deployment/
It is said you have to run :
$ java -jar target/dt-v1.0.0.jar /path/to/env.yml /path/to/server.yml
to build the java package, I think you have to run :
$ mvn package -DskipTests
Data tools is the only Conveyal product still using the old C VEX URL pattern. We should update it so our VEX nginx proxy doesn't need to contain URL rewrite rules. Alternatively update everything to use the old VEX URL pattern which @mattwigway pointed out is probably better since it uses named URL parameters.
The reconcilePatternStops method in the sql-editor branch fails when a pattern is duplicated or reversed. This is due to an IndexOutOfBounds exception when attempting to iterate over the "original" set of pattern stops.
The method at hand is really only intended to reconcile changes to pattern stops with trips and their stop times, so a good way to preempt this failure would be to return before these failures if and only if there are no trips for the pattern. Any rearrangement of pattern stops should be permitted as long as there are no trips that could be jumbled by the result.
Otherwise, we should run the conditions in a try/catch and return a useful message about the ambiguous/complex rearrangement of pattern stops and its effect on stop times.
Due to missing validation check in catalogueglobal/datatools-ui#43, an NPE is thrown during snapshot export at ScheduleException::serviceRunsOn:
java.lang.NullPointerException
at com.conveyal.datatools.editor.models.transit.ScheduleException.serviceRunsOn(ScheduleException.java:67)
at com.conveyal.datatools.editor.jobs.ProcessGtfsSnapshotExport.run(ProcessGtfsSnapshotExport.java:155)
at com.conveyal.datatools.editor.controllers.api.SnapshotController.writeSnapshotAsGtfs(SnapshotController.java:325)
at com.conveyal.datatools.editor.controllers.api.SnapshotController.downloadSnapshotWithToken(SnapshotController.java:380)
at spark.RouteImpl$1.handle(RouteImpl.java:61)
at spark.http.matching.Routes.execute(Routes.java:61)
at spark.http.matching.MatcherFilter.doFilter(MatcherFilter.java:126)
at spark.embeddedserver.jetty.JettyHandler.doHandle(JettyHandler.java:50)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:189)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
at org.eclipse.jetty.server.Server.handle(Server.java:517)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:242)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:75)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:213)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:745)
Auth0 user management features do not appear to work for "new" Auth0 clients/tenants. The previous method to generate an Auth0 management API token has been replaced with a token generation process that expired tokens after 24 hours.
When fetching or mutating users, the back end should use something similar to this. One issue encountered in an initial attempt to fix this was that the grant_type: client_credentials nor the suggested alternatives client_secret_post or client_secret_basic were able to be used for the new Auth0 tenant being tested.
It appears that in a recent refactor, an important step in validating that the trip stopTimes match the pattern stops was ignored when creating new trips. The step that confirms matching stop IDs between these two sets needs to be added to TripController::validateTrip.
We experienced an issue where thousands of connections were being held open to Auth0. A sample connection appeared like this after running lsof:
TCP ip-172-30-2-68.eu-west-1.compute.internal:49970->ec2-52-28-229-49.eu-central-1.compute.amazonaws.com:https
This was preventing the application from making any other http requests and the result to users was that the user token could not be verified. If encountered in the future, it may be possible to kill off these TCP connections or simply restarting the server will work.
A potential fix might be to use the auth0 java library for getting the user profile in Auth0Connection.java, which may manage auth0 connections better.
Some actions need two http successive requests to be performed. Downloading a feed is an example of such an action. The first request download a token '/api/manager/${SecureOrPublic}/feedversion/${FeedVersion.id}/downloadtoken' then the second request uses this token to download the feed /api/manager/downloadfeed/${DownloadedToken}. The token is available for only one dowload, which causes the download of feeds fails for clients behind a security proxy that plays any http request twice. A first time to test the server and a second time to serve the original requester.
14:58:30.111 [main] INFO c.c.datatools.manager.DataManager - Loading env.yml: configurations/default/env.yml
14:58:30.115 [main] INFO c.c.datatools.manager.DataManager - Loading server.yml: configurations/default/server.yml
14:58:30.191 [main] INFO c.c.d.manager.persistence.DataStore - /Users/zbouziane/Data/cataloguedev/mapdb/projects
14:58:30.375 [main] INFO c.c.datatools.manager.DataManager - Initializing gtfs-api for bucket cataloguedev/gtfs/ and cache dir /Users/zbouziane/Data/cataloguedev/gtfs
log4j:WARN No appenders could be found for logger (com.amazonaws.AmazonWebServiceClient).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
14:58:31.078 [main] INFO com.conveyal.gtfs.GTFSCache - Using bucket cataloguedev for GTFS Cache
14:58:31.109 [Thread-1] INFO org.eclipse.jetty.util.log - Logging initialized @6536ms
14:58:31.178 [Thread-1] INFO s.e.jetty.EmbeddedJettyServer - == Spark has ignited ...
14:58:31.179 [Thread-1] INFO s.e.jetty.EmbeddedJettyServer - >> Listening on 0.0.0.0:4000
14:58:31.182 [Thread-1] INFO org.eclipse.jetty.server.Server - jetty-9.3.z-SNAPSHOT
14:58:31.313 [Thread-1] INFO o.e.jetty.server.ServerConnector - Started ServerConnector@d6e9a92{HTTP/1.1,[http/1.1]}{0.0.0.0:4000}
14:58:31.316 [Thread-1] INFO org.eclipse.jetty.server.Server - Started @6744ms
14:58:31.388 [main] WARN c.c.d.m.c.api.GtfsApiController - No extension provided for GTFS API
14:58:31.390 [pool-1-thread-1] INFO c.c.d.manager.jobs.FeedUpdater - Fetching feeds...
14:58:31.390 [pool-1-thread-1] INFO c.c.d.manager.jobs.FeedUpdater - Current eTag list {}
14:58:31.405 [main] WARN c.c.datatools.manager.DataManager - Config property modules.gtfsplus.enabled not found
14:58:31.405 [main] WARN c.c.datatools.manager.DataManager - Config property modules.gtfsplus.enabled not found
14:58:31.405 [main] WARN c.c.datatools.manager.DataManager - Config property modules.gtfsplus.enabled not found
14:58:31.413 [main] WARN c.c.datatools.manager.DataManager - Config property modules.dump.enabled not found
14:58:31.413 [main] WARN c.c.datatools.manager.DataManager - Config property modules.dump.enabled not found
14:58:31.413 [main] WARN c.c.datatools.manager.DataManager - Config property modules.dump.enabled not found
14:58:31.417 [main] WARN c.c.datatools.manager.DataManager - Config property extensions.mtc.enabled not found
14:58:31.418 [main] WARN c.c.datatools.manager.DataManager - Config property extensions.mtc.enabled not found
14:58:31.418 [main] WARN c.c.datatools.manager.DataManager - Config property extensions.mtc.enabled not found
14:58:31.418 [main] INFO c.c.datatools.manager.DataManager - Registering TransitLand Resource
14:58:31.419 [main] INFO c.c.datatools.manager.DataManager - Registering TransitFeeds Resource
14:58:31.974 [pool-1-thread-1] INFO c.c.d.manager.jobs.FeedUpdater - No feeds updated...
14:58:42.352 [qtp891452239-15] INFO c.c.d.m.c.api.ProjectController - found projects: 0
/api/manager/public/region
14:58:42.439 [qtp891452239-17] INFO c.c.d.manager.persistence.DataStore - /Users/zbouziane/Data/cataloguedev/mapdb/region
14:58:45.460 [qtp891452239-20] INFO c.c.d.m.c.api.ProjectController - found projects: 0
14:58:51.244 [qtp891452239-18] INFO c.c.d.m.c.api.ProjectController - found projects: 0
14:59:00.327 [qtp891452239-21] INFO c.c.d.manager.persistence.DataStore - /Users/zbouziane/Data/cataloguedev/mapdb/feedsources
14:59:00.429 [qtp891452239-20] INFO c.c.d.m.c.api.ProjectController - found projects: 1
developer.digitalfactory@gmail.com
setting fs project
14:59:17.457 [qtp891452239-17] INFO c.c.d.manager.persistence.DataStore - /Users/zbouziane/Data/cataloguedev/mapdb/feedversions
14:59:17.500 [qtp891452239-17] INFO c.c.d.manager.persistence.DataStore - /Users/zbouziane/Data/cataloguedev/mapdb/externalFeedSourceProperties
developer.digitalfactory@gmail.com
14:59:44.113 [qtp891452239-20] INFO c.c.d.m.c.api.FeedVersionController - Saving feed from upload <FeedSource tampere (10e9ec1c-a6c7-418e-95f9-9eb4d911f167)>
14:59:44.113 [qtp891452239-20] INFO c.c.d.manager.persistence.FeedStore - Writing file to /Users/zbouziane/Data/cataloguedev/gtfs/tampere-20170418T145943+02-10e9ec1c-a6c7-418e-95f9-9eb4d911f167.zip
14:59:44.182 [qtp891452239-20] INFO c.c.d.manager.persistence.FeedStore - Copying version to latest <FeedSource tampere (10e9ec1c-a6c7-418e-95f9-9eb4d911f167)>
14:59:44.200 [qtp891452239-20] INFO c.c.d.manager.models.FeedVersion - New GTFS file saved: tampere-20170418T145943+02-10e9ec1c-a6c7-418e-95f9-9eb4d911f167.zip
14:59:44.209 [qtp891452239-20] INFO c.c.d.m.c.api.FeedVersionController - Last modified: Fri Dec 02 09:44:10 CET 2016
14:59:44.214 [qtp891452239-20] INFO c.c.d.manager.persistence.FeedStore - Downloading feed from s3
14:59:44.366 [qtp891452239-20] ERROR c.c.d.manager.persistence.FeedStore - Error downloading from s3
com.amazonaws.services.s3.model.AmazonS3Exception: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: 25A928491624EB60), S3 Extended Request ID: DM7fWaJ+9bKZcoXEBrjDDBS8nwwgxQwoCek7HdEtky+y1QQxr0VnfF/B1HnZxz48gl6fPkY1h60=
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1579)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1249)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:742)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:716)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4191)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4138)
at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1385)
at com.conveyal.datatools.manager.persistence.FeedStore.getFeed(FeedStore.java:159)
at com.conveyal.datatools.manager.models.FeedVersion.getGtfsFile(FeedVersion.java:155)
at com.conveyal.datatools.manager.models.FeedVersion.hash(FeedVersion.java:441)
at com.conveyal.datatools.manager.controllers.api.FeedVersionController.createFeedVersion(FeedVersionController.java:134)
at spark.ResponseTransformerRouteImpl$1.handle(ResponseTransformerRouteImpl.java:47)
at spark.http.matching.Routes.execute(Routes.java:61)
at spark.http.matching.MatcherFilter.doFilter(MatcherFilter.java:126)
at spark.embeddedserver.jetty.JettyHandler.doHandle(JettyHandler.java:50)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:189)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
at org.eclipse.jetty.server.Server.handle(Server.java:517)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:242)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:75)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:213)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:745)
14:59:44.989 [qtp891452239-20] INFO c.c.d.e.j.ProcessGtfsSnapshotMerge - GTFS Snapshot Merge for feedVersion tampere-20170418T145943+02-10e9ec1c-a6c7-418e-95f9-9eb4d911f167.zip
14:59:44.991 [Thread-11] INFO c.c.d.manager.jobs.ValidateFeedJob - Running ValidateFeedJob for tampere-20170418T145943+02-10e9ec1c-a6c7-418e-95f9-9eb4d911f167.zip
14:59:44.993 [Thread-11] INFO c.c.d.manager.auth.Auth0Users - Auth0 getUsers URL=https://cataloguetest.eu.auth0.com/api/v2/users/auth0%7C57a1f0c5b8155e90194e5087
14:59:45.233 [Thread-11] INFO com.conveyal.gtfs.GTFSCache - Attempting to download cached GTFS MapDB.
14:59:45.368 [Thread-11] WARN com.conveyal.gtfs.GTFSCache - DB file for key gtfs/tampere-20170418T145943-02-10e9ec1c-a6c7-418e-95f9-9eb4d911f167 does not exist on S3.
14:59:45.368 [Thread-11] INFO com.conveyal.gtfs.GTFSCache - Loading feed from local cache directory...
14:59:45.368 [Thread-11] INFO com.conveyal.gtfs.GTFSCache - Feed not found locally, downloading from S3.
14:59:45.399 [Thread-11] ERROR com.conveyal.gtfs.GTFSCache - Could not download feed at s3://cataloguedev/gtfs/tampere-20170418T145943-02-10e9ec1c-a6c7-418e-95f9-9eb4d911f167.
14:59:45.400 [Thread-11] WARN c.c.d.manager.models.FeedVersion - No GTFS feed exists for version: tampere-20170418T145943+02-10e9ec1c-a6c7-418e-95f9-9eb4d911f167.zip
Building network
14:59:45.443 [Thread-12] INFO c.c.d.e.j.ProcessGtfsSnapshotMerge - Cleared old data
14:59:45.444 [Thread-12] INFO com.conveyal.gtfs.GTFSCache - Attempting to download cached GTFS MapDB.
14:59:45.478 [Thread-12] WARN com.conveyal.gtfs.GTFSCache - DB file for key gtfs/tampere-20170418T145943-02-10e9ec1c-a6c7-418e-95f9-9eb4d911f167 does not exist on S3.
14:59:45.478 [Thread-12] INFO com.conveyal.gtfs.GTFSCache - Loading feed from local cache directory...
14:59:45.478 [Thread-12] INFO com.conveyal.gtfs.GTFSCache - Feed not found locally, downloading from S3.
14:59:45.506 [Thread-12] ERROR com.conveyal.gtfs.GTFSCache - Could not download feed at s3://cataloguedev/gtfs/tampere-20170418T145943-02-10e9ec1c-a6c7-418e-95f9-9eb4d911f167.
java.lang.NullPointerException
at com.conveyal.datatools.manager.models.FeedVersion.getGtfsFeed(FeedVersion.java:181)
at com.conveyal.datatools.editor.jobs.ProcessGtfsSnapshotMerge.run(ProcessGtfsSnapshotMerge.java:120)
at java.lang.Thread.run(Thread.java:745)
Exception in thread "Thread-12" spark.HaltException
at spark.Service.halt(Service.java:437)
at spark.Spark.halt(Spark.java:854)
at com.conveyal.datatools.editor.jobs.ProcessGtfsSnapshotMerge.run(ProcessGtfsSnapshotMerge.java:459)
at java.lang.Thread.run(Thread.java:745)When I insert a stop into a trip pattern, the database will create a null stopTime (see https://github.com/catalogueglobal/datatools-server/blob/dev/src/main/java/com/conveyal/datatools/editor/models/transit/TripPattern.java#L189). When this stop time tries to get editing in the UI, an error happens.
The backend should correctly insert the stop and add a non-null stop time. I propose the default stopTime creation behavior:
The download request '/api/manager/downloadfeed/${token}' when succeeding return a zip file with a randomly generated name. In the response header there is "Content-Disposition: attachment; filename=test4474876194626471181.zip". Is it possible to return the right filename instead?
Currently (and since the earliest versions of gtfs-editor), when importing a feed into the editor the calendar_dates.txt table is entirely disregarded. Schedule exceptions must be generated and managed from scratch within the feed editor. This is due in part to the complex combinations of calendars.txt and calendar_dates.txt that can be found in GTFS feeds. One notable example, a case which is also described in the reference documentation, is the TriMet feed, which explicitly defines all dates of operation in calendar_dates.txt.
While there is no support currently for moving from calendar_dates.txt to editor-defined schedule exceptions, there may be some simple rules we could introduce to simply convert single or paired dates entries into the SWAP exception type.
Currently the frontend talks directly to gtfs-api without going through the backend, and with no authentication. This means the contents of all feeds are accessible to someone who knows the architecture of the system or sniffs the traffic.
In the MonitorableJob class we have an enum listing the types of monitorable jobs. However monitorable jobs are all represented by subclasses of the MonitorableJob class. This enum is redundant. Each subclass should either override a function returning a unique string, or (perhaps better) the shared superclass should just define a public function returning the simple class name so it's visible when the instances are serialized to JSON.
Especially in FeedVersionController::register, FeedSourceController::register, ProjectController::register and UserController::register.
We are seeing slowdowns that may be caused by insufficient heap space. Investigating this, we find that there is a bug/feature in Spark web framework that causes all uploads to be buffered in memory. It has been fixed in the development version of Spark, but not yet released.
perwendel/spark#891
Whether or not this is the cause of our slowdowns, it should be addressed. We don't want to be needlessly buffering 10 copies of the Netherlands feed in memory at any given time.
One option is to await the next release of Spark. Another is to not use Spark, assuming all the functionality we need is in the underlying Jetty library.
That warning sounds pretty serve:
bash-4.4# cd /opt/datatools-server/ && mvn package -DskipTests
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for com.conveyal:datatools-server:jar:3.0.0-SNAPSHOT
[WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-jar-plugin is missing. @ line 85, column 21
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
pom.xml currently looks like this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<mainClass>com.conveyal.datatools.manager.DataManager</mainClass>
</manifest>
</archive>
</configuration>
</plugin>Is this possible? I've been trying to run the mvn package command on the code from a Docker container but kept running into MongoDB issues, so I've opted to build without testing. Unfortunately I now get a "no main manifest attribute" error when I try to run the built JAR file. Do the tests have to have been successfully run for the manifest file to be generated?
With use_s3_storage set to false:
17/04/20 12:14:10.621 [Thread-14] WARN com.conveyal.gtfs.BaseGTFSCache - Feed tampere-20170420T121408+02-f6a79e00-b150-4725-b385-712826a46351 not found locally
java.lang.NullPointerException
at com.conveyal.datatools.manager.models.FeedVersion.getGtfsFeed(FeedVersion.java:182)
at com.conveyal.datatools.editor.jobs.ProcessGtfsSnapshotMerge.run(ProcessGtfsSnapshotMerge.java:130)
at java.lang.Thread.run(Thread.java:745)
Use ApiMain::registerFeedSource to write feed version files to local file system.
A feed version for the Atlanta Streetcar had no service and null bounds. An NPE (reproduced below) was encountered when loading loading a deployment that referenced this version due to line 448.
Jan 29 14:52:32.482 [qtp1046044319-2877] ERROR c.c.datatools.manager.DataManager - error
com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.NullPointerException) (through reference chain: com.conveyal.datatools.manager.models.Deployment["projectBounds"])
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:210) ~[dt-latest-dev.jar:na]
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:177) ~[dt-latest-dev.jar:na]
at com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.java:190) ~[dt-latest-dev.jar:na]
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:674) ~[dt-latest-dev.jar:na]
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:156) ~[dt-latest-dev.jar:na]
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:129) ~[dt-latest-dev.jar:na]
at com.fasterxml.jackson.databind.ObjectWriter._configAndWriteValue(ObjectWriter.java:1052) ~[dt-latest-dev.jar:na]
at com.fasterxml.jackson.databind.ObjectWriter.writeValueAsString(ObjectWriter.java:923) ~[dt-latest-dev.jar:na]
at com.conveyal.datatools.manager.utils.json.JsonManager.write(JsonManager.java:74) ~[dt-latest-dev.jar:na]
at spark.ResponseTransformerRouteImpl$1.render(ResponseTransformerRouteImpl.java:42) ~[dt-latest-dev.jar:na]
at spark.http.matching.Routes.execute(Routes.java:62) ~[dt-latest-dev.jar:na]
at spark.http.matching.MatcherFilter.doFilter(MatcherFilter.java:126) ~[dt-latest-dev.jar:na]
at spark.embeddedserver.jetty.JettyHandler.doHandle(JettyHandler.java:50) ~[dt-latest-dev.jar:na]
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:189) ~[dt-latest-dev.jar:na]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[dt-latest-dev.jar:na]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119) ~[dt-latest-dev.jar:na]
at org.eclipse.jetty.server.Server.handle(Server.java:517) ~[dt-latest-dev.jar:na]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) ~[dt-latest-dev.jar:na]
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:242) ~[dt-latest-dev.jar:na]
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261) ~[dt-latest-dev.jar:na]
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) ~[dt-latest-dev.jar:na]
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:75) ~[dt-latest-dev.jar:na]
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:213) ~[dt-latest-dev.jar:na]
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147) ~[dt-latest-dev.jar:na]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) ~[dt-latest-dev.jar:na]
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) ~[dt-latest-dev.jar:na]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_131]
Caused by: java.lang.NullPointerException: null
at java.awt.geom.Rectangle2D$Double.setRect(Rectangle2D.java:421) ~[na:1.8.0_131]
at com.conveyal.datatools.manager.models.Deployment.getProjectBounds(Deployment.java:448) ~[dt-latest-dev.jar:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_131]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_131]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_131]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_131]
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:536) ~[dt-latest-dev.jar:na]
at com.fasterxml.jackson.databind.ser.impl.FilteredBeanPropertyWriter$SingleView.serializeAsField(FilteredBeanPropertyWriter.java:64) ~[dt-latest-dev.jar:na]
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:666) ~[dt-latest-dev.jar:na]
... 23 common frames omitted
Back end enhancements to deployments:
Related to catalogueglobal/datatools-ui#18.
@demory experienced this behavior when fetching feeds for the DC area. Only three out of 4 or 5 feeds were fetched after clicking 'Fetch All'.
$ mvn test
...
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.conveyal.datatools.editor.ProcessGtfsSnapshotMergeTest
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/zbouziane/.m2/repository/ch/qos/logback/logback-classic/1.1.3/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/zbouziane/.m2/repository/org/slf4j/slf4j-simple/1.7.10/slf4j-simple-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
16:03:53,041 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
16:03:53,041 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
16:03:53,042 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [file:/Users/zbouziane/tmp/tmp/datatools-server/target/classes/logback.xml]
16:03:53,044 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs multiple times on the classpath.
16:03:53,044 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [jar:file:/Users/zbouziane/.m2/repository/com/conveyal/osm-lib/1.1.0-SNAPSHOT/osm-lib-1.1.0-SNAPSHOT.jar!/logback.xml]
16:03:53,044 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [file:/Users/zbouziane/tmp/tmp/datatools-server/target/classes/logback.xml]
16:03:53,158 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
16:03:58,171 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
16:03:58,178 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
16:03:58,203 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
16:03:58,254 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to DEBUG
16:03:58,254 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
16:03:58,254 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
16:03:58,255 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@22eeefeb - Registering current configuration as safe fallback point
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
17/04/19 16:03:58.263 [main] INFO com.conveyal.datatools.DatatoolsTest - DatatoolsTest setup
17/04/19 16:03:58.461 [main] INFO c.c.datatools.manager.DataManager - Loading env.yml: ../configurations/gtfs.works/dev/settings.yml
17/04/19 16:03:58.462 [main] INFO c.c.datatools.manager.DataManager - Loading server.yml: ../configurations/gtfs.works/dev/server.yml
java.io.FileNotFoundException: ../configurations/gtfs.works/dev/settings.yml (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at com.conveyal.datatools.manager.DataManager.loadConfig(DataManager.java:307)
at com.conveyal.datatools.manager.DataManager.main(DataManager.java:78)
at com.conveyal.datatools.DatatoolsTest.setUp(DatatoolsTest.java:25)
at com.conveyal.datatools.LoadFeedTest.setUp(LoadFeedTest.java:28)
at com.conveyal.datatools.editor.ProcessGtfsSnapshotMergeTest.setUp(ProcessGtfsSnapshotMergeTest.java:35)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
17/04/19 16:03:58.466 [main] INFO com.conveyal.datatools.LoadFeedTest - ProcessGtfsSnapshotMergeTest setup
17/04/19 16:03:58.479 [main] WARN c.c.datatools.manager.DataManager - Config property application.data.mapdb not found
...
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.