Option to store properties as a single JSON column

[simonw]

Option to store properties as a single JSON column

[simonw]

Tests failed there https://github.com/simonw/geojson-to-sqlite/runs/5416550417?check_suite_focus=true because dirty-equals isn't available for Python 3.6 - I'm going to drop 3.6 and add 3.10.

[simonw]

Sounds like a good feature to me - a command-line option which means "bundle all of the properties into JSON in a column called properties" perhaps.

Current options are:

~ % geojson-to-sqlite --help
Usage: geojson-to-sqlite [OPTIONS] DB_PATH TABLE GEOJSON

  Import GeoJSON into a SQLite database

Options:
  --nl                   Use newline-delimited GeoJSON features
  --pk TEXT              Column to use as a primary key
  --alter                Add any missing columns
  --spatialite           Use SpatiaLite
  --spatial-index        Create spatial indexes
  --spatialite_mod TEXT  Path to SpatiaLite module, for if --spatialite cannot
                         find it automatically
  --help                 Show this message and exit.

I'm having trouble thinking of a good name for the new option. A few ideas:

• geojson-to-sqlite ... --properties=properties
• geojson-to-sqlite ... --bundle=properties
• geojson-to-sqlite ... --json-properties=properties

I think I like the first best, but I'm not entirely convinced by any of them.

[eyeseast]

I think --properties or --properties=properties is totally reasonable. Maybe doing geojson-to-sqlite ... --properties defaults to a field called properties and --properties=props calls the field props.

[simonw]

I don't actually know of a pattern using Click that can understand the difference between --properties and --properties=properties - I think because Click treats --properties=x and --properties x the same, so options that take values always expect the next token to be a value.

[eyeseast]

Looks like this should do it: https://click.palletsprojects.com/en/8.0.x/options/#optional-value

I can try it out tomorrow and send a PR.

[simonw]

Huh, TIL! Didn't think that was possible.

Looking forward to the PR!

[eyeseast]

Two related PRs that might be worth getting merged before this:

• #26 because it's probably the same block of code and will likely cause a conflict with this
• simonw/sqlite-utils#385 consolidates the spatialite operations this library uses into sqlite-utils.gis

That second one is less critical, if you're prioritizing.

[simonw]

Tests failed after the merge for some reason, re-opening. https://github.com/simonw/geojson-to-sqlite/runs/5416463816?check_suite_focus=true

Looks to me like it's a floating point comparison problem:

E         -  {'geometry': '{"type":"Polygon","coordinates":[[[-8.0859375,60.93043220292333],[-16.875,50.28933925329177],[-5.9765625,48.92249926375825],[4.21875,52.26815737376816],[1.0546875,60.06484046010452],[-8.0859375,60.93043220292333]]]}',
E         +  {'geometry': '{"type":"Polygon","coordinates":[[[-8.0859375,60.93043220292332],[-16.875,50.28933925329177],[-5.9765625,48.92249926375824],[4.21875,52.26815737376816],[1.0546875,60.06484046010452],[-8.0859375,60.93043220292332]]]}',

[simonw]

I've been looking for an excuse to try https://dirty-equals.helpmanual.io/usage/