Feature request: table stats to doublecheck successful import

[weecology]

Feature request: table stats to doublecheck successful import

[ethanwhite]

I definitely like the idea and running a quick query for the counts of every table should be pretty straight forward.

Ben and I have also previously talked about doing something a bit more comprehensive where we basically dump the tables to files and get checksums for the files to confirm that they contain exactly the same data as a version that we believe to be a valid import based on having looked at the files post import. This is easy and efficient in both MySQL (SELECT INTO OUTFILE) and Postgres (COPY) and it looks like maybe for SQLite as well (.mode csv).

The one issue here is that if the data gets updated it will flag as being different, but I think that we can handle that with a combination of a clear description to the user (it's not necessarily that it's wrong, it's just different so you should probably take a look) and having a cron job on a server that checks for changes in databases periodically and pings us if there's a change so that we can update the checksums.

On the other hand, this might get unnecessarily complicated... Thoughts?

[ethanwhite]

Closing since this clearly hasn't risen to the top of the list. We do have some plans for implementing checksums as mentioned above.