derhuerst/berlin-gtfs-rt-server

Expose Berlin & Brandenburg transit data as a GTFS-RT feed.

berlin-gtfs-rt-server's Issues

Question: Do I realy need to provide the GTFS Schedule feed at 2 places?

Hi!
I was wondering if I really need to provide the GTFS Schedule feed at 2 places.

In the database:

berlin-gtfs-rt-server/build.sh

Line 9 in 7e9e095

-P gtfs -N 'https://vbb-gtfs.jannisr.de/latest/'

Via URL:

berlin-gtfs-rt-server/start.sh

Line 29 in 7e9e095

--feed-url 'https://vbb-gtfs.jannisr.de/latest/' \

Is there a workaround for GTFS Schedule feeds that ar not provided via URL?

Cheers!

Observation: unrecognized value "on;" for "ON_ERROR_STOP": Boolean expected

Hi folks,
Have you ever noticed the unrecognized value "on;" for "ON_ERROR_STOP": Boolean expected feedback when calling npm run build? Here is a larger snippet.

...
ALTER TABLE
COPY 53855
COMMIT
+++ realpath ./build.sh
++ dirname /home/pacs/sib00/users/gtfsr_srv_vbb/git/berlin-gtfs-rt-server/build.sh
+ lib=/home/pacs/sib00/users/gtfsr_srv_vbb/git/berlin-gtfs-rt-server/lib
+ sponge
+ psql -b
+ NODE_ENV=production
+ node_modules/.bin/build-gtfs-match-index /home/pacs/sib00/users/gtfsr_srv_vbb/git/berlin-gtfs-rt-server/lib/hafas-info.js /home/pacs/sib00/users/gtfsr_srv_vbb/git/berlin-gtfs-rt-server/lib/gtfs-info.js
stops
routes
trip headsigns
unrecognized value "on;" for "ON_ERROR_STOP": Boolean expected
BEGIN
CREATE TABLE

COPY 579515
CREATE INDEX
...

Cheers!

npm run build fails: sponge missing

Hi there,
According to this line I called this instruction

npm run build

, that came back with this reply.

npm run build

> [email protected] build
> ./build.sh && npm run docs

+ grep '^PG'
PGPORT=<secret>
PGPASSWORD=<secret>
PGUSER=<secret>
PGDATABASE=<secret>
PGHOST=<secret>
+ NODE_ENV=production
+ node_modules/.bin/gtfs-to-sql -d --trips-without-shape-id --routes-without-agency-id -- gtfs/agency.csv gtfs/calendar.csv gtfs/calendar_dates.csv gtfs/frequencies.csv gtfs/routes.csv gtfs/stop_times.csv gtfs/stops.csv gtfs/transfers.csv gtfs/trips.csv
+ psql -b
+ sponge
./build.sh: line 24: sponge: command not found
PostGraphile PostgreSQL user's password: <secret>
PostrREST PostgreSQL user's password: <secret>

I am wondering about this

./build.sh: line 24: sponge: command not found

and this

PostGraphile PostgreSQL user's password: <secret>
PostrREST PostgreSQL user's password: <secret>

snippet. Any hint why sponge is not found and the GTFS feed is not imported into PostgreSQL?

Cheers!

Question: Is there a checklist how to adjust this repository to a different GTFS Schedule feed & Hafas client?

Hi!
Thank you so much for providing and maintaining this repository.

I forked this repository to evaluate if it is possible to replicate it for a different GTFS Schedule feed and Hafas client.

I am wondering how a checklist could look like that provide an overview for the task ahead. I found the following places that are specific to https://github.com/derhuerst/berlin-gtfs-rt-server and need an adjustment.

• Provide GTFS Schedule feed to database:

  berlin-gtfs-rt-server/build.sh

  Line 7 in 7e9e095

  wget --compression auto \

  )
• Provide URL to GTFS Schedule feed:

  berlin-gtfs-rt-server/start.sh

  Line 29 in 7e9e095

  --feed-url 'https://vbb-gtfs.jannisr.de/latest/' \

  )

• berlin-gtfs-rt-server/package.json

  Line 48 in 7e9e095

  "normalize-vbb-station-name-for-search": "^1.0.0",

• How/Where do I provide the specific Hafas client?

Whats missing?

Cheers!

allow importing into a custom DB schema

Edit @derhuerst:
The gist of this thread is here: #9 (comment)
gtfs-via-postgres's ability to import into a custom schema should be exposed, and match-gtfs-rt-to-gtfs should be adapted to support this too. Then, a schema switch can be added in here.

depends on derhuerst/match-gtfs-rt-to-gtfs#8

---

Hi folks,
I am wondering if someone has already successfully replicated this repository using the --schema switch.

I am failing with this feedback.

$ time npm run build

> [email protected] build
> ./build.sh && npm run docs

+ wget --compression auto -r --no-parent --no-directories -R .csv.gz -P gtfs -N https://vbb-gtfs.jannisr.de/latest/
--2024-02-12 13:08:57--  https://vbb-gtfs.jannisr.de/latest/
Resolving vbb-gtfs.jannisr.de (vbb-gtfs.jannisr.de)... 2001:41d0:701:1100::26d, 54.37.75.136
Connecting to vbb-gtfs.jannisr.de (vbb-gtfs.jannisr.de)|2001:41d0:701:1100::26d|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘gtfs/index.html’

index.html                                 [ <=>                                                                         ]     624  --.-KB/s    in 0s      

Last-modified header missing -- time-stamps turned off.
2024-02-12 13:08:57 (6.35 MB/s) - ‘gtfs/index.html’ saved [3222]

Loading robots.txt; please ignore errors.
--2024-02-12 13:08:57--  https://vbb-gtfs.jannisr.de/robots.txt
...

--2024-02-12 13:08:57--  https://vbb-gtfs.jannisr.de/latest/stops.csv
Reusing existing connection to [vbb-gtfs.jannisr.de]:443.
HTTP request sent, awaiting response... 304 Not Modified
File ‘gtfs/stops.csv’ not modified on server. Omitting download.

...

FINISHED --2024-02-12 13:08:57--
Total wall clock time: 0.4s
Downloaded: 1 files, 624 in 0s (6.35 MB/s)
+ env
+ grep '^PG'
...
+ NODE_ENV=production
+ node_modules/.bin/gtfs-to-sql -d --schema vbb --trips-without-shape-id --routes-without-agency-id -- gtfs/agency.csv gtfs/calendar.csv gtfs/calendar_dates.csv gtfs/frequencies.csv gtfs/routes.csv gtfs/stop_times.csv gtfs/stops.csv gtfs/transfers.csv gtfs/trips.csv
+ psql -b
+ sponge
is_valid_lang_code
is_timezone
calendar
  processed 2892 rows
calendar_dates
  processed 97563 rows
service_days
routes
  processed 1271 rows
trips
  processed 241801 rows
frequencies
  processed 0 rows
agency
  processed 33 rows
stops
  processed 41413 rows
stop_times
  processed 5748473 rows
transfers
  processed 53855 rows
NOTICE:  extension "postgis" already exists, skipping
CREATE EXTENSION
CREATE SCHEMA
BEGIN
CREATE FUNCTION
DO
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE TYPE
CREATE TABLE
COPY 2892
CREATE TYPE
CREATE TABLE
COPY 97563
CREATE INDEX
CREATE INDEX
SELECT 269243
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE TYPE
CREATE TABLE
COPY 1271
CREATE INDEX
CREATE TYPE
CREATE TYPE
CREATE TABLE
COPY 241801
CREATE TYPE
CREATE TABLE
COPY 0
CREATE INDEX
CREATE INDEX
CREATE TABLE
COPY 33
CREATE TYPE
CREATE TYPE
CREATE TABLE
COPY 41413
ALTER TABLE
CREATE INDEX
CREATE TYPE
CREATE TYPE
CREATE TABLE
COPY 5748473
CREATE INDEX
CREATE INDEX
UPDATE 5748473
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE VIEW
CREATE FUNCTION
CREATE VIEW
CREATE FUNCTION
CREATE TYPE
CREATE TABLE
ALTER TABLE
COPY 53855
COMMIT
+++ realpath ./build.sh
++ dirname /home/pacs/sib00/users/gtfsr_srv_vbb/git/berlin-gtfs-rt-server/build.sh
+ lib=/home/pacs/sib00/users/gtfsr_srv_vbb/git/berlin-gtfs-rt-server/lib
+ NODE_ENV=production
+ node_modules/.bin/build-gtfs-match-index /home/pacs/sib00/users/gtfsr_srv_vbb/git/berlin-gtfs-rt-server/lib/hafas-info.js /home/pacs/sib00/users/gtfsr_srv_vbb/git/berlin-gtfs-rt-server/lib/gtfs-info.js
+ psql -b
+ sponge
stops
error: relation "stops" does not exist
    at Parser.parseErrorMessage (/home/pacs/sib00/users/gtfsr_srv_vbb/git/berlin-gtfs-rt-server/node_modules/pg-protocol/dist/parser.js:287:98)
    at Parser.handlePacket (/home/pacs/sib00/users/gtfsr_srv_vbb/git/berlin-gtfs-rt-server/node_modules/pg-protocol/dist/parser.js:126:29)
    at Parser.parse (/home/pacs/sib00/users/gtfsr_srv_vbb/git/berlin-gtfs-rt-server/node_modules/pg-protocol/dist/parser.js:39:38)
    at Socket.<anonymous> (/home/pacs/sib00/users/gtfsr_srv_vbb/git/berlin-gtfs-rt-server/node_modules/pg-protocol/dist/index.js:11:42)
    at Socket.emit (node:events:517:28)
    at addChunk (node:internal/streams/readable:368:12)
    at readableAddChunk (node:internal/streams/readable:341:9)
    at Readable.push (node:internal/streams/readable:278:10)
    at TCP.onStreamRead (node:internal/stream_base_commons:190:23) {
  length: 105,
  severity: 'ERROR',
  code: '42P01',
  detail: undefined,
  hint: undefined,
  position: '297',
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'parse_relation.c',
  line: '1392',
  routine: 'parserOpenTable'
}
unrecognized value "on;" for "ON_ERROR_STOP": Boolean expected
BEGIN
ERROR:  relation "stops" does not exist
STATEMENT:  CREATE TABLE stops_stable_ids (
	stop_id TEXT NOT NULL,
	FOREIGN KEY (stop_id) REFERENCES stops,
	stable_id TEXT NOT NULL,
	specificity INTEGER NOT NULL
);
ERROR:  current transaction is aborted, commands ignored until end of transaction block
STATEMENT:  COPY stops_stable_ids FROM STDIN csv;

I do not observe this behavior when I omit the --schema switch.

Cheers!

correct content-type?

The TriMet TripUpdates GTFS-RT feed uses application/grtfeed:

curl 'https://developer.trimet.org/ws/V1/TripUpdate?appID=…' --head

HTTP/1.1 200 
Date: Tue, 22 Dec 2020 16:35:28 GMT
Content-Type: application/grtfeed
Content-Length: 39500
Access-Control-Allow-Origin: *
Connection: close
Strict-Transport-Security: max-age=31536000

use station DHIDs from HAFAS for matching

das VBB-HAFAS gibt sogar, in einem extra-feld versteckt, die stations-DHID zurück, also bspw. für 900029371/S+U Rathaus Spandau (Berlin) [Bus Seegef. Str.] die stations-DHID de:11000:900029371, aber nicht die DHID der Haltestelle, also z.b. de:11000:900029371::1.

Error [ERR_MODULE_NOT_FOUND]: Cannot find package '@derhuerst/technical-docs-cli'

Dear you,
When calling npm run build have you ever met this reply?

...
"277889210","westvictorbur grundschule victorbur"
"277889213","zob"
"277889219","zob"
"277889209","suedbrookmerland moorhusen igs"

> [email protected] docs
> ./build-docs.js

node:internal/errors:496
    ErrorCaptureStackTrace(err);
    ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find package '@derhuerst/technical-docs-cli' imported from /home/pacs/sib00/users/gtfsr_srv_fp/git/fp-gtfs-rt-server/build-docs.js
    at new NodeError (node:internal/errors:405:5)
    at packageResolve (node:internal/modules/esm/resolve:916:9)
    at moduleResolve (node:internal/modules/esm/resolve:973:20)
    at defaultResolve (node:internal/modules/esm/resolve:1193:11)
    at ModuleLoader.defaultResolve (node:internal/modules/esm/loader:403:12)
    at ModuleLoader.resolve (node:internal/modules/esm/loader:372:25)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:249:38)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:76:39)
    at link (node:internal/modules/esm/module_job:75:36) {
  code: 'ERR_MODULE_NOT_FOUND'
}

Node.js v18.19.0
...

Cheers!