andrewharvey/asgs2pgsql

Scripts to load the ABS ASGS into PostgreSQL (with PostGIS geometries)

asgs2pgsql's Issues

make it easier to run

ideas include:

an interactive wizard which walks through step by step. creating the db, choosing a schema name, choosing which datasets to load

full release each year?

we need to decide what to do with the odd year releases. eg. 2013, 2015, 2017. ie. where volume 2 or 3 data is updated but volume 1 is not.

Where should the references link to?

Option 1

Each year include the full dataset, ie. use Volume 1 from 2016 and push this into the asgs_2017 schema.

Option 2

Reference back to the last year. eg. volume 3 from 2017 would reference asgs_2016 volume 1. asgs_2016 volume 1 would always reference asgs_2016 volume 3.

tester implementation and test cases

1. Set up a mechanism to run tests on the data.
2. Write test cases.

load geometry as GDA94 LL by default and make spherical mercator an optional extra

use COPY to load csv data rather than inserts

04-load-csv-data.pl could instead of executing the INSERT query, could produce a .copy file which could be loaded at the end using the a COPY query. This should speed it up.

use PostgreSQL COMMENTs

add PostgreSQL COMMENTs for functions defined by the loader