Giter Site home page Giter Site logo

mobilitydb / mobilitydb-berlinmod Goto Github PK

View Code? Open in Web Editor NEW
8.0 4.0 5.0 161.57 MB

BerlinMOD benchmark, a data generator and a tool for comparying Moving Object Databases

PLpgSQL 98.00% Shell 0.96% Dockerfile 1.04%
benchmarking mobilitydb postgresql postgis

mobilitydb-berlinmod's Introduction

Build Status Coverage Status Codacy Badge Gitter

MobilityDB

An open source geospatial trajectory data management & analysis platform

MobilityDB Logo

MobilityDB is a database management system for moving object geospatial trajectories, such as GPS traces. It adds support for temporal and spatio-temporal objects to the PostgreSQL database and its spatial extension PostGIS.

MobilityDB is developed by the Computer & Decision Engineering Department of the Université libre de Bruxelles (ULB) under the direction of Prof. Esteban Zimányi. ULB is an OGC Associate Member and member of the OGC Moving Feature Standard Working Group (MF-SWG).

OGC Associate Member Logo

The MobilityDB project is managed by a steering committee.

More information about MobilityDB, including publications, presentations, etc., can be found in the MobilityDB website.

Documentation

The pregenerated user and developer documentation can be found here.

To generate the documentation locally, refer to our documentation generation section

Benefits

Experimental Projects

These projects push the boundaries of MobilityDB and connect it with the PostgreSQL/PostGIS ecosystem.

Cloud

Visualization

Public Transport

Mailing Lists

There are two mailing lists for MobilityDB hosted on OSGeo mailing list server:

For general questions and topics about how to use MobilityDB, please write to the user mailing list.

Branches

  • The master branch has the development of the next release.
  • The stable-major.minor branches have the patch releases of the correspong major.minor release.

The complete list of releases is available here

Requirements

  • Linux (other UNIX-like systems may work, but remain untested)
  • CMake >= 3.7
  • PostgreSQL >= 12
  • PostGIS >= 3.0
  • GEOS >= 3.8
  • PROJ4 >= 6.1
  • JSON-C
  • GNU Scientific Library (GSL)
  • Development files for PostgreSQL, PostGIS, GEOS, PROJ4, JSON-C, GSL

For example, you can build the following command to install all MobilityDB build dependencies for Debian-based systems using PostgreSQL 16 and PostGIS 3:

apt install build-essential cmake postgresql-server-dev-16 libgeos-dev libproj-dev libjson-c-dev libgsl-dev

Building & Installation

Here is the gist:

git clone https://github.com/MobilityDB/MobilityDB
mkdir MobilityDB/build
cd MobilityDB/build
cmake ..
make
sudo make install

You should also set the following in postgresql.conf depending on the version of PostGIS you have installed (below we use PostGIS 3):

shared_preload_libraries = 'postgis-3'
max_locks_per_transaction = 128

If you do not preload the PostGIS library you will not be able to load the MobilityDB library and will get an error message such as the following one

ERROR:  could not load library "/usr/local/pgsql/lib/libMobilityDB-1.0.so": undefined symbol: ST_Distance

Notice that you can find the location of the postgresql.conf file as given next:

$ which postgres
/usr/local/pgsql/bin/postgres
$ ls /usr/local/pgsql/data/postgresql.conf
/usr/local/pgsql/data/postgresql.conf

As can be seen, the PostgreSQL binaries are in the bin subdirectory while the postgresql.conf file is in the data subdirectory.

Once MobilityDB is installed, it needs to be enabled in each database you want to use it in. In the example below we use a database named mobility.

createdb mobility
psql mobility -c "CREATE EXTENSION PostGIS"
psql mobility -c "CREATE EXTENSION MobilityDB"

Docker Container

Docker images with MobilityDB and all its dependencies are available here. These images are based on the official Postgres and Postgis docker images, please refer to them for more information.

If you have docker installed in your system you can run:

docker pull mobilitydb/mobilitydb
docker volume create mobilitydb_data
docker run --name mobilitydb -e POSTGRES_PASSWORD=mysecretpassword \
  -p 25432:5432 -v mobilitydb_data:/var/lib/postgresql -d mobilitydb/mobilitydb
psql -h localhost -p 25432 -U postgres

The first command is to download the latest most up-to-date image of MobilityDB. The second command creates a volume container on the host, that we will use to persist the PostgreSQL database files outside of the MobilityDB container. The third command executes this binary image of PostgreSQL, PostGIS, and MobilityDB with the TCP port 5432 in the container mapped to port 25432 on the Docker host (user = postgres, db = postgres, pw=mysecretpassword). The fourth command is to connect to the database using psql.

Note that you can define the environment variable PGPASSWORD to avoid an interactive pw prompt.

PGPASSWORD=mysecretpassword psql -h localhost -p 25432 -U postgres

Issues

Please report any issues you may have.

Generating the Documentation

Pregenerated versions of the user and developer documentation can be found here.

User's Manual

You can generate the user's manual in HTML, PDF, and EPUB formats. The manual is generated in English and in other available languages (currently only Spanish). For this, it is necessary to specify appropriate options in the cmake command as follows:

  • DOC_ALL: Generate in HTML, PDF, and EPUB formats
  • DOC_HTML: Generate in HTML format
  • DOC_PDF: Generate in PDF format
  • DOC_EPUB: Generate in EPUB format
  • LANG_ALL: Generate in all available languages
  • ES: Generate the Spanish documentation

For example, the following command generates the documentation in all formats and in all languages.

cmake -D DOC_ALL=true -D LANG_ALL=true ..
make doc

As another example, the following command generates the English documentation in PDF.

cmake -D DOC_PDF=true ..
make doc

The resulting documentation will be generated in the doc directory of the build directory.

Developer's Documentation

You can generate the English developer's documentation in HTML format. For this, it is necessary to the option DOC_DEV in the cmake command as follows:

cmake -D DOC_DEV=true ..
make doc_dev

The resulting HTML documentation will be generated in the doxygen directory of the build directory.

Licenses

mobilitydb-berlinmod's People

Contributors

estebanzimanyi avatar lorenc1o avatar mahmsakr avatar mbakli avatar sajouaou avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

mobilitydb-berlinmod's Issues

Error setting Geom column in Points table.

When I am loading the berlinMOD data from the given CSV, I get an error when I run the script:

UPDATE Points
SET Geom = ST_Transform(ST_SetSRID(ST_MakePoint(PosX, PosY), 4326), 5676);

The error:

ERROR:  transform: latitude or longitude exceeded limits (-14)

hello, a question about BerlinMOD

Hi, Thank you for creating MobilityDB , This is a great project.
But I encountered some problems in the process of use:

I executed berlinmod_runall.sh, But no Cars, TripsInput, Instants, Periods, Points, RegionsInput, Regions, and Trips are generated.

postgres@ubuntu:/home/mca/Desktop/data$ sh berlinmod_runall.sh
Password:
Password:
Password for user postgres:
NOTICE: installing required extension "postgis"
CREATE EXTENSION
Password for user postgres:
CREATE EXTENSION
Execution starts at: Wed May 12 06:38:48 2021


       COMMAND LINE CONFIGURATION             *

Filename = brussels.osm
Configuration file = mapconfig_brussels.xml
host = localhost
port = 5432
dbname = brussels
username = postgres
schema=
prefix =
suffix =
Don't drop tables
Don't create indexes
Don't add OSM nodes


Testing database connection: brussels
database connection successful: brussels
Connecting to the database
connection success

Creating tables...
TABLE: ways_vertices_pgr created ... OK.
TABLE: ways created ... OK.
TABLE: pointsofinterest created ... OK.
TABLE: configuration created ... OK.
Opening configuration file: mapconfig_brussels.xml
Parsing configuration

Exporting configuration ...

  • Done
    Counting lines ...
  • Done
    Opening data file: brussels.osm total lines: 9565284
    Parsing data

End Of file

Finish Parsing data

Adding auxiliary tables to database...

Export Ways ...
Processing 432717 ways:
[| ] (4%) Total processed: 20000 Vertices inserted: 32633 Split ways inserted 31654
[| ] (9%) Total processed: 40000 Vertices inserted: 7037 Split ways inserted 8134
[
| ] (13%) Total processed: 60000 Vertices inserted: 4185 Split ways inserted 5060
[| ] (18%) Total processed: 80000 Vertices inserted: 4006 Split ways inserted 5132
[
| ] (23%) Total processed: 100000 Vertices inserted: 3150 Split ways inserted 4182
[| ] (27%) Total processed: 120000 Vertices inserted: 178 Split ways inserted 220
[
| ] (32%) Total processed: 140000 Vertices inserted: 131 Split ways inserted 240
[
| ] (36%) Total processed: 160000 Vertices inserted: 102 Split ways inserted 150
[
| ] (41%) Total processed: 180000 Vertices inserted: 59 Split ways inserted 95
[
| ] (46%) Total processed: 200000 Vertices inserted: 334 Split ways inserted 586
[| ] (50%) Total processed: 220000 Vertices inserted: 268 Split ways inserted 408
[
| ] (55%) Total processed: 240000 Vertices inserted: 467 Split ways inserted 677
[
| ] (60%) Total processed: 260000 Vertices inserted: 339 Split ways inserted 492
[
| ] (64%) Total processed: 280000 Vertices inserted: 504 Split ways inserted 769
[| ] (69%) Total processed: 300000 Vertices inserted: 534 Split ways inserted 876
[| ] (73%) Total processed: 320000 Vertices inserted: 1951 Split ways inserted 3057
[
| ] (78%) Total processed: 340000 Vertices inserted: 2075 Split ways inserted 3463
[
| ] (83%) Total processed: 360000 Vertices inserted: 3398 Split ways inserted 5511
[| ] (87%) Total processed: 380000 Vertices inserted: 2325 Split ways inserted 3764
[
| ] (92%) Total processed: 400000 Vertices inserted: 1644 Split ways inserted 2954
[
| ] (97%) Total processed: 420000 Vertices inserted: 1523 Split ways inserted 2602
[
*********************|] (100%) Total processed: 432717 Vertices inserted: 1582 Split ways inserted 2784

Creating indexes ...

Processing Points of Interest ...
#########################
size of streets: 432717
Execution started at: Wed May 12 06:38:48 2021
Execution ended at: Wed May 12 06:39:12 2021
Elapsed time: 24.113 Seconds.
User CPU time: -> 10.0416 seconds
#########################
osm2pgsql version 1.2.1 (64 bit id space)

Password:
Allocating memory for dense node cache
Allocating dense node cache in one big chunk
Allocating memory for sparse node cache
Sharing dense sparse
Node-cache: cache=800MB, maxblocks=12800*65536, allocation method=3
Using built-in tag processing pipeline
Using projection SRS 3857 (Spherical Mercator)
Setting up table: planet_osm_point
Setting up table: planet_osm_line
Setting up table: planet_osm_polygon
Setting up table: planet_osm_roads

Reading in file: postgres
node cache: stored: 0(-nan%), storage efficiency: -nan% (dense blocks: 0, sparse nodes: 0), hit rate: -nan%
Osm2pgsql failed due to ERROR: Cannot detect file format. Try using -r.
berlinmod_runall.sh: 16: cannot create brussels.tar: Permission denied
pg_restore: connecting to database for restore
pg_restore: creating DATABASE "brussels"
pg_restore: [archiver (db)] Error while PROCESSING TOC:
pg_restore: [archiver (db)] Error from TOC entry 7649; 1262 120467 DATABASE brussels postgres
pg_restore: [archiver (db)] could not execute query: ERROR: database "brussels" already exists
Command was: CREATE DATABASE brussels WITH TEMPLATE = template0 ENCODING = 'UTF8' LC_COLLATE = 'en_US.UTF-8' LC_CTYPE = 'en_US.UTF-8';

pg_restore: connecting to new database "brussels"
pg_restore: creating EXTENSION "postgis"
pg_restore: creating COMMENT "EXTENSION postgis"
pg_restore: creating EXTENSION "mobilitydb"
pg_restore: creating COMMENT "EXTENSION mobilitydb"
pg_restore: creating EXTENSION "pgrouting"
pg_restore: creating COMMENT "EXTENSION pgrouting"
pg_restore: creating TABLE "public.configuration"
pg_restore: [archiver (db)] Error from TOC entry 223; 1259 126199 TABLE configuration postgres
pg_restore: [archiver (db)] could not execute query: ERROR: relation "configuration" already exists
Command was: CREATE TABLE public.configuration (
id integer NOT NULL,
tag_id integer,
tag_key text,
tag_value text,
priority double precision,
maxspeed double precision,
maxspeed_forward double precision,
maxspeed_backward double precision,
force character(1)
)
WITH (autovacuum_enabled='false');

pg_restore: creating SEQUENCE "public.configuration_id_seq"
pg_restore: [archiver (db)] Error from TOC entry 222; 1259 126197 SEQUENCE configuration_id_seq postgres
pg_restore: [archiver (db)] could not execute query: ERROR: relation "configuration_id_seq" already exists
Command was: CREATE SEQUENCE public.configuration_id_seq
AS integer
START WITH 1
INCREMENT BY 1
NO MINVALUE
NO MAXVALUE
CACHE 1;

pg_restore: creating SEQUENCE OWNED BY "public.configuration_id_seq"
pg_restore: creating TABLE "public.planet_osm_line"
pg_restore: [archiver (db)] Error from TOC entry 225; 1259 126549 TABLE planet_osm_line postgres
pg_restore: [archiver (db)] could not execute query: ERROR: relation "planet_osm_line" already exists
Command was: CREATE UNLOGGED TABLE public.planet_osm_line (
osm_id bigint,
access text,
"addr:housename" text,
"addr:housenumber" text,
"addr:interpolation" text,
admin_level text,
aerialway text,
aeroway text,
amenity text,
area text,
barrier text,
bicycle text,
brand text,
bridge text,
boundary text,
building text,
construction text,
covered text,
culvert text,
cutting text,
denomination text,
disused text,
embankment text,
foot text,
"generator:source" text,
harbour text,
highway text,
historic text,
horse text,
intermittent text,
junction text,
landuse text,
layer text,
leisure text,
lock text,
man_made text,
military text,
motorcar text,
name text,
"natural" text,
office text,
oneway text,
operator text,
place text,
population text,
power text,
power_source text,
public_transport text,
railway text,
ref text,
religion text,
route text,
service text,
shop text,
sport text,
surface text,
toll text,
tourism text,
"tower:type" text,
tracktype text,
tunnel text,
water text,
waterway text,
wetland text,
width text,
wood text,
z_order integer,
way_area real,
way public.geometry(LineString,3857)
)
WITH (autovacuum_enabled='false');

pg_restore: creating TABLE "public.planet_osm_point"
pg_restore: [archiver (db)] Error from TOC entry 224; 1259 126543 TABLE planet_osm_point postgres
pg_restore: [archiver (db)] could not execute query: ERROR: relation "planet_osm_point" already exists
Command was: CREATE UNLOGGED TABLE public.planet_osm_point (
osm_id bigint,
access text,
"addr:housename" text,
"addr:housenumber" text,
"addr:interpolation" text,
admin_level text,
aerialway text,
aeroway text,
amenity text,
area text,
barrier text,
bicycle text,
brand text,
bridge text,
boundary text,
building text,
capital text,
construction text,
covered text,
culvert text,
cutting text,
denomination text,
disused text,
ele text,
embankment text,
foot text,
"generator:source" text,
harbour text,
highway text,
historic text,
horse text,
intermittent text,
junction text,
landuse text,
layer text,
leisure text,
lock text,
man_made text,
military text,
motorcar text,
name text,
"natural" text,
office text,
oneway text,
operator text,
place text,
population text,
power text,
power_source text,
public_transport text,
railway text,
ref text,
religion text,
route text,
service text,
shop text,
sport text,
surface text,
toll text,
tourism text,
"tower:type" text,
tunnel text,
water text,
waterway text,
wetland text,
width text,
wood text,
z_order integer,
way public.geometry(Point,3857)
)
WITH (autovacuum_enabled='false');

pg_restore: creating TABLE "public.planet_osm_polygon"
pg_restore: [archiver (db)] Error from TOC entry 226; 1259 126555 TABLE planet_osm_polygon postgres
pg_restore: [archiver (db)] could not execute query: ERROR: relation "planet_osm_polygon" already exists
Command was: CREATE UNLOGGED TABLE public.planet_osm_polygon (
osm_id bigint,
access text,
"addr:housename" text,
"addr:housenumber" text,
"addr:interpolation" text,
admin_level text,
aerialway text,
aeroway text,
amenity text,
area text,
barrier text,
bicycle text,
brand text,
bridge text,
boundary text,
building text,
construction text,
covered text,
culvert text,
cutting text,
denomination text,
disused text,
embankment text,
foot text,
"generator:source" text,
harbour text,
highway text,
historic text,
horse text,
intermittent text,
junction text,
landuse text,
layer text,
leisure text,
lock text,
man_made text,
military text,
motorcar text,
name text,
"natural" text,
office text,
oneway text,
operator text,
place text,
population text,
power text,
power_source text,
public_transport text,
railway text,
ref text,
religion text,
route text,
service text,
shop text,
sport text,
surface text,
toll text,
tourism text,
"tower:type" text,
tracktype text,
tunnel text,
water text,
waterway text,
wetland text,
width text,
wood text,
z_order integer,
way_area real,
way public.geometry(Geometry,3857)
)
WITH (autovacuum_enabled='false');

pg_restore: creating TABLE "public.planet_osm_roads"
pg_restore: [archiver (db)] Error from TOC entry 227; 1259 126561 TABLE planet_osm_roads postgres
pg_restore: [archiver (db)] could not execute query: ERROR: relation "planet_osm_roads" already exists
Command was: CREATE UNLOGGED TABLE public.planet_osm_roads (
osm_id bigint,
access text,
"addr:housename" text,
"addr:housenumber" text,
"addr:interpolation" text,
admin_level text,
aerialway text,
aeroway text,
amenity text,
area text,
barrier text,
bicycle text,
brand text,
bridge text,
boundary text,
building text,
construction text,
covered text,
culvert text,
cutting text,
denomination text,
disused text,
embankment text,
foot text,
"generator:source" text,
harbour text,
highway text,
historic text,
horse text,
intermittent text,
junction text,
landuse text,
layer text,
leisure text,
lock text,
man_made text,
military text,
motorcar text,
name text,
"natural" text,
office text,
oneway text,
operator text,
place text,
population text,
power text,
power_source text,
public_transport text,
railway text,
ref text,
religion text,
route text,
service text,
shop text,
sport text,
surface text,
toll text,
tourism text,
"tower:type" text,
tracktype text,
tunnel text,
water text,
waterway text,
wetland text,
width text,
wood text,
z_order integer,
way_area real,
way public.geometry(LineString,3857)
)
WITH (autovacuum_enabled='false');

pg_restore: creating TABLE "public.pointsofinterest"
pg_restore: [archiver (db)] Error from TOC entry 221; 1259 126190 TABLE pointsofinterest postgres
pg_restore: [archiver (db)] could not execute query: ERROR: relation "pointsofinterest" already exists
Command was: CREATE TABLE public.pointsofinterest (
pid bigint NOT NULL,
osm_id bigint,
vertex_id bigint,
edge_id bigint,
side character(1),
fraction double precision,
length_m double precision,
tag_name text,
tag_value text,
name text,
the_geom public.geometry(Point,4326),
new_geom public.geometry(Point,4326)
)
WITH (autovacuum_enabled='false');

pg_restore: creating SEQUENCE "public.pointsofinterest_pid_seq"
pg_restore: [archiver (db)] Error from TOC entry 220; 1259 126188 SEQUENCE pointsofinterest_pid_seq postgres
pg_restore: [archiver (db)] could not execute query: ERROR: relation "pointsofinterest_pid_seq" already exists
Command was: CREATE SEQUENCE public.pointsofinterest_pid_seq
START WITH 1
INCREMENT BY 1
NO MINVALUE
NO MAXVALUE
CACHE 1;

pg_restore: creating SEQUENCE OWNED BY "public.pointsofinterest_pid_seq"
pg_restore: creating TABLE "public.ways"
pg_restore: [archiver (db)] Error from TOC entry 219; 1259 126180 TABLE ways postgres
pg_restore: [archiver (db)] could not execute query: ERROR: relation "ways" already exists
Command was: CREATE TABLE public.ways (
gid bigint NOT NULL,
osm_id bigint,
tag_id integer,
length double precision,
length_m double precision,
name text,
source bigint,
target bigint,
source_osm bigint,
target_osm bigint,
cost double precision,
reverse_cost double precision,
cost_s double precision,
reverse_cost_s double precision,
rule text,
one_way integer,
oneway text,
x1 double precision,
y1 double precision,
x2 double precision,
y2 double precision,
maxspeed_forward double precision,
maxspeed_backward double precision,
priority double precision DEFAULT 1,
the_geom public.geometry(LineString,4326)
)
WITH (autovacuum_enabled='false');

pg_restore: creating SEQUENCE "public.ways_gid_seq"
pg_restore: [archiver (db)] Error from TOC entry 218; 1259 126178 SEQUENCE ways_gid_seq postgres
pg_restore: [archiver (db)] could not execute query: ERROR: relation "ways_gid_seq" already exists
Command was: CREATE SEQUENCE public.ways_gid_seq
START WITH 1
INCREMENT BY 1
NO MINVALUE
NO MAXVALUE
CACHE 1;

pg_restore: creating SEQUENCE OWNED BY "public.ways_gid_seq"
pg_restore: creating TABLE "public.ways_vertices_pgr"
pg_restore: [archiver (db)] Error from TOC entry 217; 1259 126171 TABLE ways_vertices_pgr postgres
pg_restore: [archiver (db)] could not execute query: ERROR: relation "ways_vertices_pgr" already exists
Command was: CREATE TABLE public.ways_vertices_pgr (
id bigint NOT NULL,
osm_id bigint,
eout integer,
lon numeric(11,8),
lat numeric(11,8),
cnt integer,
chk integer,
ein integer,
the_geom public.geometry(Point,4326)
)
WITH (autovacuum_enabled='false');

pg_restore: creating SEQUENCE "public.ways_vertices_pgr_id_seq"
pg_restore: [archiver (db)] Error from TOC entry 216; 1259 126169 SEQUENCE ways_vertices_pgr_id_seq postgres
pg_restore: [archiver (db)] could not execute query: ERROR: relation "ways_vertices_pgr_id_seq" already exists
Command was: CREATE SEQUENCE public.ways_vertices_pgr_id_seq
START WITH 1
INCREMENT BY 1
NO MINVALUE
NO MAXVALUE
CACHE 1;

pg_restore: creating SEQUENCE OWNED BY "public.ways_vertices_pgr_id_seq"
pg_restore: creating DEFAULT "public.configuration id"
pg_restore: creating DEFAULT "public.pointsofinterest pid"
pg_restore: creating DEFAULT "public.ways gid"
pg_restore: creating DEFAULT "public.ways_vertices_pgr id"
pg_restore: processing data for table "public.configuration"
pg_restore: [archiver (db)] Error from TOC entry 7639; 0 126199 TABLE DATA configuration postgres
pg_restore: [archiver (db)] COPY failed for table "configuration": ERROR: duplicate key value violates unique constraint "configuration_pkey"
DETAIL: Key (id)=(1) already exists.
CONTEXT: COPY configuration, line 1
pg_restore: processing data for table "public.planet_osm_line"
pg_restore: processing data for table "public.planet_osm_point"
pg_restore: processing data for table "public.planet_osm_polygon"
pg_restore: processing data for table "public.planet_osm_roads"
pg_restore: processing data for table "public.pointsofinterest"
pg_restore: processing data for table "public.spatial_ref_sys"
pg_restore: processing data for table "public.ways"
pg_restore: [archiver (db)] Error from TOC entry 7635; 0 126180 TABLE DATA ways postgres
pg_restore: [archiver (db)] COPY failed for table "ways": ERROR: duplicate key value violates unique constraint "ways_pkey"
DETAIL: Key (gid)=(1) already exists.
CONTEXT: COPY ways, line 1
pg_restore: processing data for table "public.ways_vertices_pgr"
pg_restore: [archiver (db)] Error from TOC entry 7633; 0 126171 TABLE DATA ways_vertices_pgr postgres
pg_restore: [archiver (db)] COPY failed for table "ways_vertices_pgr": ERROR: duplicate key value violates unique constraint "ways_vertices_pgr_pkey"
DETAIL: Key (id)=(1) already exists.
CONTEXT: COPY ways_vertices_pgr, line 1
pg_restore: executing SEQUENCE SET configuration_id_seq
pg_restore: executing SEQUENCE SET pointsofinterest_pid_seq
pg_restore: executing SEQUENCE SET ways_gid_seq
pg_restore: executing SEQUENCE SET ways_vertices_pgr_id_seq
pg_restore: creating CONSTRAINT "public.configuration configuration_pkey"
pg_restore: [archiver (db)] Error from TOC entry 7496; 2606 126502 CONSTRAINT configuration configuration_pkey postgres
pg_restore: [archiver (db)] could not execute query: ERROR: multiple primary keys for table "configuration" are not allowed
Command was: ALTER TABLE ONLY public.configuration
ADD CONSTRAINT configuration_pkey PRIMARY KEY (id);

pg_restore: creating CONSTRAINT "public.configuration configuration_tag_id_key"
pg_restore: [archiver (db)] Error from TOC entry 7498; 2606 126504 CONSTRAINT configuration configuration_tag_id_key postgres
pg_restore: [archiver (db)] could not execute query: ERROR: relation "configuration_tag_id_key" already exists
Command was: ALTER TABLE ONLY public.configuration
ADD CONSTRAINT configuration_tag_id_key UNIQUE (tag_id);

pg_restore: creating CONSTRAINT "public.pointsofinterest pointsofinterest_osm_id_key"
pg_restore: [archiver (db)] Error from TOC entry 7491; 2606 126542 CONSTRAINT pointsofinterest pointsofinterest_osm_id_key postgres
pg_restore: [archiver (db)] could not execute query: ERROR: relation "pointsofinterest_osm_id_key" already exists
Command was: ALTER TABLE ONLY public.pointsofinterest
ADD CONSTRAINT pointsofinterest_osm_id_key UNIQUE (osm_id);

pg_restore: creating CONSTRAINT "public.pointsofinterest pointsofinterest_pkey"
pg_restore: [archiver (db)] Error from TOC entry 7493; 2606 126539 CONSTRAINT pointsofinterest pointsofinterest_pkey postgres
pg_restore: [archiver (db)] could not execute query: ERROR: multiple primary keys for table "pointsofinterest" are not allowed
Command was: ALTER TABLE ONLY public.pointsofinterest
ADD CONSTRAINT pointsofinterest_pkey PRIMARY KEY (pid);

pg_restore: creating CONSTRAINT "public.ways ways_pkey"
pg_restore: [archiver (db)] Error from TOC entry 7488; 2606 126511 CONSTRAINT ways ways_pkey postgres
pg_restore: [archiver (db)] could not execute query: ERROR: multiple primary keys for table "ways" are not allowed
Command was: ALTER TABLE ONLY public.ways
ADD CONSTRAINT ways_pkey PRIMARY KEY (gid);

pg_restore: creating CONSTRAINT "public.ways_vertices_pgr ways_vertices_pgr_osm_id_key"
pg_restore: [archiver (db)] Error from TOC entry 7483; 2606 126508 CONSTRAINT ways_vertices_pgr ways_vertices_pgr_osm_id_key postgres
pg_restore: [archiver (db)] could not execute query: ERROR: relation "ways_vertices_pgr_osm_id_key" already exists
Command was: ALTER TABLE ONLY public.ways_vertices_pgr
ADD CONSTRAINT ways_vertices_pgr_osm_id_key UNIQUE (osm_id);

pg_restore: creating CONSTRAINT "public.ways_vertices_pgr ways_vertices_pgr_pkey"
pg_restore: [archiver (db)] Error from TOC entry 7485; 2606 126506 CONSTRAINT ways_vertices_pgr ways_vertices_pgr_pkey postgres
pg_restore: [archiver (db)] could not execute query: ERROR: multiple primary keys for table "ways_vertices_pgr" are not allowed
Command was: ALTER TABLE ONLY public.ways_vertices_pgr
ADD CONSTRAINT ways_vertices_pgr_pkey PRIMARY KEY (id);

pg_restore: creating INDEX "public.pointsofinterest_the_geom_idx"
pg_restore: [archiver (db)] Error from TOC entry 7494; 1259 126540 INDEX pointsofinterest_the_geom_idx postgres
pg_restore: [archiver (db)] could not execute query: ERROR: relation "pointsofinterest_the_geom_idx" already exists
Command was: CREATE INDEX pointsofinterest_the_geom_idx ON public.pointsofinterest USING gist (the_geom);

pg_restore: creating INDEX "public.ways_the_geom_idx"
pg_restore: [archiver (db)] Error from TOC entry 7489; 1259 126537 INDEX ways_the_geom_idx postgres
pg_restore: [archiver (db)] could not execute query: ERROR: relation "ways_the_geom_idx" already exists
Command was: CREATE INDEX ways_the_geom_idx ON public.ways USING gist (the_geom);

pg_restore: creating INDEX "public.ways_vertices_pgr_the_geom_idx"
pg_restore: [archiver (db)] Error from TOC entry 7486; 1259 126509 INDEX ways_vertices_pgr_the_geom_idx postgres
pg_restore: [archiver (db)] could not execute query: ERROR: relation "ways_vertices_pgr_the_geom_idx" already exists
Command was: CREATE INDEX ways_vertices_pgr_the_geom_idx ON public.ways_vertices_pgr USING gist (the_geom);

pg_restore: creating FK CONSTRAINT "public.ways ways_source_fkey"
pg_restore: [archiver (db)] Error from TOC entry 7499; 2606 126512 FK CONSTRAINT ways ways_source_fkey postgres
pg_restore: [archiver (db)] could not execute query: ERROR: constraint "ways_source_fkey" for relation "ways" already exists
Command was: ALTER TABLE ONLY public.ways
ADD CONSTRAINT ways_source_fkey FOREIGN KEY (source) REFERENCES public.ways_vertices_pgr(id);

pg_restore: creating FK CONSTRAINT "public.ways ways_source_osm_fkey"
pg_restore: [archiver (db)] Error from TOC entry 7501; 2606 126522 FK CONSTRAINT ways ways_source_osm_fkey postgres
pg_restore: [archiver (db)] could not execute query: ERROR: constraint "ways_source_osm_fkey" for relation "ways" already exists
Command was: ALTER TABLE ONLY public.ways
ADD CONSTRAINT ways_source_osm_fkey FOREIGN KEY (source_osm) REFERENCES public.ways_vertices_pgr(osm_id);

pg_restore: creating FK CONSTRAINT "public.ways ways_tag_id_fkey"
pg_restore: [archiver (db)] Error from TOC entry 7503; 2606 126532 FK CONSTRAINT ways ways_tag_id_fkey postgres
pg_restore: [archiver (db)] could not execute query: ERROR: constraint "ways_tag_id_fkey" for relation "ways" already exists
Command was: ALTER TABLE ONLY public.ways
ADD CONSTRAINT ways_tag_id_fkey FOREIGN KEY (tag_id) REFERENCES public.configuration(tag_id);

pg_restore: creating FK CONSTRAINT "public.ways ways_target_fkey"
pg_restore: [archiver (db)] Error from TOC entry 7500; 2606 126517 FK CONSTRAINT ways ways_target_fkey postgres
pg_restore: [archiver (db)] could not execute query: ERROR: constraint "ways_target_fkey" for relation "ways" already exists
Command was: ALTER TABLE ONLY public.ways
ADD CONSTRAINT ways_target_fkey FOREIGN KEY (target) REFERENCES public.ways_vertices_pgr(id);

pg_restore: creating FK CONSTRAINT "public.ways ways_target_osm_fkey"
pg_restore: [archiver (db)] Error from TOC entry 7502; 2606 126527 FK CONSTRAINT ways ways_target_osm_fkey postgres
pg_restore: [archiver (db)] could not execute query: ERROR: constraint "ways_target_osm_fkey" for relation "ways" already exists
Command was: ALTER TABLE ONLY public.ways
ADD CONSTRAINT ways_target_osm_fkey FOREIGN KEY (target_osm) REFERENCES public.ways_vertices_pgr(osm_id);

WARNING: errors ignored on restore: 31
Password for user postgres:
psql:brussels_preparedata.sql:27: NOTICE: table "edges" does not exist, skipping
DROP TABLE
SELECT 82810
psql:brussels_preparedata.sql:37: NOTICE: table "nodes" does not exist, skipping
DROP TABLE
SELECT 66781
CREATE INDEX
CREATE INDEX
CREATE INDEX
UPDATE 82810
DELETE 1984
CREATE INDEX
CREATE INDEX
psql:brussels_preparedata.sql:83: NOTICE: table "communes" does not exist, skipping
DROP TABLE
SELECT 19
psql:brussels_preparedata.sql:108: NOTICE: table "communesgeom" does not exist, skipping
DROP TABLE
SELECT 0
UPDATE 0
DELETE 0
ALTER TABLE
UPDATE 0
ALTER TABLE
UPDATE 19
psql:brussels_preparedata.sql:171: NOTICE: table "saintjosse" does not exist, skipping
DROP TABLE
DROP TABLE
psql:brussels_preparedata.sql:176: NOTICE: table "homeregions" does not exist, skipping
DROP TABLE
SELECT 19
CREATE INDEX
psql:brussels_preparedata.sql:185: NOTICE: table "workregions" does not exist, skipping
DROP TABLE
SELECT 19
CREATE INDEX
psql:brussels_preparedata.sql:194: NOTICE: table "homenodes" does not exist, skipping
DROP TABLE
SELECT 0
CREATE INDEX
psql:brussels_preparedata.sql:202: NOTICE: table "worknodes" does not exist, skipping
DROP TABLE
SELECT 0
CREATE INDEX
Password for user postgres:
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
psql:berlinmod_datagenerator.sql:309: NOTICE: function berlinmod_roadcategory() does not exist, skipping
DROP FUNCTION
CREATE FUNCTION
psql:berlinmod_datagenerator.sql:327: NOTICE: type "step" does not exist, skipping
DROP TYPE
CREATE TYPE
psql:berlinmod_datagenerator.sql:336: NOTICE: function createpath() does not exist, skipping
DROP FUNCTION
CREATE FUNCTION
psql:berlinmod_datagenerator.sql:380: NOTICE: function createtrip() does not exist, skipping
DROP FUNCTION
CREATE FUNCTION
psql:berlinmod_datagenerator.sql:713: NOTICE: function berlinmod_selecthomenode() does not exist, skipping
DROP FUNCTION
CREATE FUNCTION
CREATE FUNCTION
psql:berlinmod_datagenerator.sql:787: NOTICE: function berlinmod_selectdestnode() does not exist, skipping
DROP FUNCTION
CREATE FUNCTION
psql:berlinmod_datagenerator.sql:817: NOTICE: function berlinmod_createlicence() does not exist, skipping
DROP FUNCTION
CREATE FUNCTION
psql:berlinmod_datagenerator.sql:843: NOTICE: function berlinmod_vehicletype() does not exist, skipping
DROP FUNCTION
CREATE FUNCTION
psql:berlinmod_datagenerator.sql:871: NOTICE: function berlinmod_vehiclemodel() does not exist, skipping
DROP FUNCTION
CREATE FUNCTION
psql:berlinmod_datagenerator.sql:901: NOTICE: function berlinmod_createtrips() does not exist, skipping
DROP FUNCTION
CREATE FUNCTION
psql:berlinmod_datagenerator.sql:1078: NOTICE: function berlinmod_generate() does not exist, skipping
DROP FUNCTION
CREATE FUNCTION
Password for user postgres:
INFO: ------------------------------------------------------------------
INFO: Starting the BerlinMOD data generator with scale factor 0.05
INFO: ------------------------------------------------------------------
INFO: Parameters:
INFO: ------------
INFO: No. of vehicles = 447, No. of days = 8, Start day = 2020-06-01
INFO: Path mode = Fastest Path, Disturb data = f
INFO: Verbosity = minimal, Trip generation = C
INFO: Execution started at 2021-05-12 06:39:34.166817-07
INFO: ------------------------------------------------------------------
INFO: Creating the Destinations table
NOTICE: table "destinations" does not exist, skipping
INFO: Creating the Vehicle, Licences, and Neighbourhood tables
NOTICE: table "vehicle" does not exist, skipping
NOTICE: table "licences" does not exist, skipping
NOTICE: table "neighbourhood" does not exist, skipping
INFO: Creating the QueryPoints and QueryRegions tables
NOTICE: table "querypoints" does not exist, skipping
NOTICE: table "queryregions" does not exist, skipping
INFO: Creating the QueryInstants and QueryPeriods tables
NOTICE: table "queryinstants" does not exist, skipping
NOTICE: table "queryperiods" does not exist, skipping
INFO: Creating the LeisureTrip table
NOTICE: table "leisuretrip" does not exist, skipping
INFO: Creation of the Paths table started at 2021-05-12 06:39:38.550869-07
NOTICE: table "paths" does not exist, skipping
INFO: Creation of the Trips table started at 2021-05-12 06:39:55.697622-07
NOTICE: table "trips" does not exist, skipping
INFO: Vehicles 1 to 100
INFO: Vehicles 101 to 200
INFO: Vehicles 201 to 300
INFO: Vehicles 301 to 400
INFO: Vehicles 401 to 447
INFO: ------------------------------------------------------------------
INFO: Execution started at 2021-05-12 06:39:34.166817-07
INFO: Execution finished at 2021-05-12 06:40:27.653865-07
INFO: Execution time 00:00:53.487048
INFO: Call to pgRouting with 4963 paths lasted 00:00:16.948261
INFO: Number of trips generated 9437
INFO: ------------------------------------------------------------------
berlinmod_generate

THE END
(1 row)

Password for user postgres:
psql:berlinmod_r_queries.sql:33: NOTICE: function berlinmod_r_queries() does not exist, skipping
DROP FUNCTION
CREATE FUNCTION

This is the whole process of code execution. I want to use BerlinMOD to benchmark MobilityDB , and some of the generated data seems to be missing? I hope you can answer my confusion in your free time

                                                                                                                                                                      Thank you.

script berlinmod_datagenerator.sql function createPath seems outdated.

Setup

pgr_version: 3.6.1

Issue:

As part of my translation of berlinmod generator to python I'm working on the functions in the berlinmod_datagenerator.sql script. When I run the function (link:

CREATE OR REPLACE FUNCTION createPath(sourceN bigint, targetN bigint, pathMode text)
) createPath (to understand it) I get the error:

temp_berlinmod=# select createPath(9598, 4010, 'Fastest Path')
temp_berlinmod-# ;
ERROR:  column p.seqno does not exist
LINE 2:     SELECT P.seqNo, P.node, P.edge
                   ^
HINT:  Perhaps you meant to reference the column "p.seq".
QUERY:  WITH Temp1 AS (
    SELECT P.seqNo, P.node, P.edge
    FROM pgr_dijkstra(query_pgr, sourceN, targetN, true) P
  ),
  Temp2 AS (
    SELECT T.seqNo,
      -- adjusting directionality
      CASE
        WHEN T.node = E.sourceNode THEN E.geom
        ELSE ST_Reverse(geom)
      END AS geom,
      maxspeed_forward AS maxSpeed, berlinmod_roadCategory(tag_id) AS category
    FROM Temp1 T, Edges E
    WHERE edge IS NOT NULL AND E.id = T.edge
  )
  SELECT array_agg((geom, maxSpeed, category)::step ORDER BY seqNo)               FROM Temp2
CONTEXT:  PL/pgSQL function createpath(bigint,bigint,text) line 14 at SQL statement

Upon further investigation I also found that indeed the return column for pgr_dijstra in version: 3.6, at least, is seq and not seqno.

Also found that use of column sourceNode in line 345 also give an error:

(psycopg2.errors.InternalError_) Column 'source' not Found
HINT:  SELECT id, sourcenode, targetNode AS target, 
                              cost_s AS cost, reverse_cost_s as reverse_cost FROM Edges
CONTEXT:  SQL function "pgr_dijkstra" statement 1

It requires the use of source and target respectively. I also checked the documentation and the expectations in the errors line up with the documentation. pg_routing documentation: https://docs.pgrouting.org/latest/en/pgr_dijkstra.html

My questions:

  1. Is this really an error or am I missing something?
  2. If this is an error can I open a pull request to update this function?

berlinmod_datagenerator.sql

Hi all,

I tried to execute a tutorial in this page https://docs.mobilitydb.com/MobilityDB-BerlinMOD/master/ch02s04.html

and all the steps before works fine, but when I try to run generator:

psql -h localhost -p 5432 -U dbowner -d brussels
-c 'select berlinmod_generate(scaleFactor := 0.005)'
-- calls the main pgplsql function to start the simulation

I have this error:

psql -h localhost -p 5433 -U postgres -d cruscotto-asset -f berlinmod_datagenerator.sql
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
DROP FUNCTION
CREATE FUNCTION
psql:berlinmod_datagenerator.sql:327: NOTICE: drop cascades to 2 other objects
DETTAGLI: drop cascades to function createpath(bigint,bigint,text)
drop cascades to function createtrip(step[],timestamp with time zone,boolean,text)
DROP TYPE
CREATE TYPE
psql:berlinmod_datagenerator.sql:336: NOTICE: function createpath() does not exist, skipping
DROP FUNCTION
CREATE FUNCTION
psql:berlinmod_datagenerator.sql:380: NOTICE: function createtrip() does not exist, skipping
DROP FUNCTION
CREATE FUNCTION
DROP FUNCTION
CREATE FUNCTION
CREATE FUNCTION
DROP FUNCTION
CREATE FUNCTION
DROP FUNCTION
CREATE FUNCTION
DROP FUNCTION
CREATE FUNCTION
DROP FUNCTION
CREATE FUNCTION
DROP FUNCTION
CREATE FUNCTION
DROP FUNCTION
CREATE FUNCTION
root@AC-ENOTARNICOLA:/home/geography/mobilityTest/DowloadFromGitHub/MobilityDB-BerlinMOD/BerlinMOD# psql -h localhost -p 5433 -U postgres -d cruscotto-asset -c 'select berlinmod_generate(scaleFactor := 0.005)'
INFO: ------------------------------------------------------------------
INFO: Starting the BerlinMOD data generator with scale factor 0.005
INFO: ------------------------------------------------------------------
INFO: Parameters:
INFO: ------------
INFO: No. of vehicles = 141, No. of days = 4, Start day = 2020-06-01
INFO: Path mode = Fastest Path, Disturb data = f
INFO: Verbosity = minimal, Trip generation = C
INFO: Execution started at 2021-02-05 14:12:24.33232+00
INFO: ------------------------------------------------------------------
INFO: Creating the Destinations table
NOTICE: table "destinations" does not exist, skipping
INFO: Creating the Vehicle, Licences, and Neighbourhood tables
NOTICE: table "vehicle" does not exist, skipping
NOTICE: table "licences" does not exist, skipping
NOTICE: table "neighbourhood" does not exist, skipping
ERROR: relation "nodes" does not exist
RIGA 1: SELECT COUNT() FROM Nodes
^
QUERY: SELECT COUNT(
) FROM Nodes
CONTESTO: PL/pgSQL function berlinmod_generate(double precision,integer,integer,date,text,text,boolean,text,text) line 185 at SQL statement

Can someone help me in order to solve it?
Thanks a lot for the great works!

Ema.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.