Consider whether we support full Tabular Data Package loading

[rufuspollock]

Consider whether we support full Tabular Data Package loading

[rufuspollock]

I'm increasingly think we at least want to support a full resources list i.e. the resources attribute in Tabular Data Package so you can build all of the tables you need and handle foreignkeys.

[danfowler]

I support this. I'd lean toward a separate tdp-sql library.

[rufuspollock]

@danfowler ok can we tdp-to-sql then (or since explicit seems better tabular-data-package-to-sql ;-) ...)

[rufuspollock]

@pudo / @danfowler how are we doing on this? I'm happy to see it in here first and then moved out or straight up in a separate repo.

/cc @pwalsh

[danfowler]

@rgrp I'm working on sorting out foreignkeys support currently. Once that's done, the full load should be trivial, I think.

[pudo]

Sounds awesome. I'd be keen to see the separation of the TDP/JTS thing since their complexity seems to differ by an order of magnitude and I think having lean JTS tooling would be a value in itself.

[pwalsh]

Definitely agree with @danfowler and @pudo on having TDP/JTS as distinct libs.

[rufuspollock]

Great we have a plan @danfowler i'll leave you to boot tabular-data-package-to-sql ;-)

I do note one subtlety: whether you want this library to handle "multiple" tables - strictly this takes you a bit into tabular data package territory but it makes sense if you have foreign keys etc. You also have the point that table naming really comes outside of JTS (it is in the resource descriptor).

[danfowler]

@rgrp tabular-data-package-to-sql will handle the loading of TDP resources, setting table names, and joining, but it will depend on jts-sql being at least able to "see" other tables.

[rufuspollock]

@danfowler makes perfect sense!

[pwalsh]

Yes, in a distinct library that is starting development now: https://github.com/okfn/datapackage-sql-py